Title: Bayesian target optimisation for high-precision holographic optogenetics
Venue: NeurIPS 2023
Accepted: yes
Keywords: Neuroscience

neural stimulation

optogenetics

calcium imaging
Abstract: Two-photon optogenetics has transformed our ability to probe the structure and function of neural circuits. However, achieving precise optogenetic control of neural ensemble activity has remained fundamentally constrained by the problem of off-target stimulation (OTS): the inadvertent activation of nearby non-target neurons due to imperfect confinement of light onto target neurons. Here we propose a novel computational approach to this problem called Bayesian target optimisation. Our approach uses nonparametric Bayesian inference to model neural responses to optogenetic stimulation, and then optimises the laser powers and optical target locations needed to achieve a desired activity pattern with minimal OTS. We validate our approach in simulations and using data from in vitro experiments, showing that Bayesian target optimisation considerably reduces OTS across all conditions we test. Together, these results establish our ability to overcome OTS, enabling optogenetic stimulation with substantially improved precision.

Bayesian target optimisation for high-precision holographic optogenetics

Abstract

Two-photon optogenetics has transformed our ability to probe the structure and function of neural circuits. However, achieving precise optogenetic control of neural ensemble activity has remained fundamentally constrained by the problem of off-target stimulation (OTS): the inadvertent activation of nearby non-target neurons due to imperfect confinement of light onto target neurons. Here we propose a novel computational approach to this problem called Bayesian target optimisation. Our approach uses nonparametric Bayesian inference to model neural responses to optogenetic stimulation, and then optimises the laser powers and optical target locations needed to achieve a desired activity pattern with minimal OTS. We validate our approach in simulations and using data from in vitro experiments, showing that Bayesian target optimisation considerably reduces OTS across all conditions we test. Together, these results establish our ability to overcome OTS, enabling optogenetic stimulation with substantially improved precision.

1 Introduction

A key technological goal in neuroscience is to gain precise control over the activity of neurons. Currently, one of the most promising approaches for achieving such control is two-photon optogenetics [1-4]. This technique relies on the use of two-photon excitation to activate opsin molecules expressed in the somatic membrane, allowing individual neurons to be targeted for stimulation. Holographic optogenetics further extends this technique by focusing two-photon excitation into ∼ 10 µ m disks of light that illuminate all of a neuron's opsin molecules in parallel [5-9]. Replicating this illumination pattern across many neurons at once then grants simultaneous optogenetic control of entire neural ensembles [10-12]. However, the spatial precision of holographic optogenetics has remained fundamentally limited by the problem of off-target stimulation (OTS): if multiple opsin-expressing neurons lie in close proximity (in the most extreme case, if their membranes are in physical contact), two-photon excitation will frequently activate opsin molecules in each neuron when attempting to stimulate only one. This can inadvertently elicit spikes in non-target neurons, compromising the precision and specificity of the optogenetic manipulation [13].

Previous attempts at overcoming OTS have been either optical or molecular in nature (e.g. by using temporal focusing [2, 6, 7, 14] or improving the soma-targeting of the opsin [15, 16, 10]), but have not yet achieved "true" single-cell precision under many experimental conditions. Frequently, experimenters will simply express opsin sparsely in a population to avoid OTS, as this reduces the number of nearby opsin-expressing neurons that could be mistakenly activated [17]. However, sparse opsin expression reduces the number of neurons that can potentially be probed during an

experiment, and therefore limits the kinds of scientific questions that can be addressed using twophoton optogenetics. Conversely, in an ideal experiment, one could precisely stimulate any neuron despite a moderate or high density of opsin-expressing cells.

Here, we develop a novel computational strategy for all-optical experiments [18-21] that reduces (and in some cases entirely eliminates) OTS (Figure 1a, b). The key insight is that heterogeneity in opsin expression and intrinsic excitability causes neurons to have different sensitivities to optical stimulation. This implies that in some cases laser power could be safely turned down to prevent the spiking of a nearby non-target neuron that is weakly driven by stimulation while still spiking a target neuron that is strongly driven by stimulation. Further, the spatial arrangement of neurons can be exploited by moving a holographic target off of a neuron's nucleus (the typical stimulation point) towards a less dense area (e.g. towards neuropil or less photosensitive neurons) while still exciting a sufficient number of the target neuron's opsin molecules to elicit a spike. While some prior experimental research has considered cell-specific tuning of laser powers [22], our advance is to identify both the optimal laser powers and target locations automatically using efficient Bayesian techniques.

To achieve highly precise optogenetic ensemble control, we propose a two-phase process called Bayesian target optimisation (Bataro). In the first phase, we map the "optogenetic receptive field" (ORF) of each neuron by stimulating at various locations around cell nuclei and at a range of laser powers to test whether they elicit spikes. To infer these ORFs without exhaustively testing every combination of location and power, we use Gaussian processes (GPs) to encode prior knowledge about how neurons respond to optogenetic stimulation. Further, we propose to use holographic stimulation to test multiple locations around one or many neurons simultaneously, ensuring that the mapping phase completes quickly. This also allows us to model how neurons integrate two-photon excitation from multiple nearby holograms. In the second phase, we then exploit knowledge of the ORFs to optimise holographic targets. We use properties of the GP to infer gradients of an objective function at a spatial resolution that exceeds the original sampling resolution, enabling us to optimise for the exact holographic targets and laser powers needed to evoke a specific neural activity pattern.

2 Related work

Our work is related to prior research in three areas: GP-based receptive field inference, optimal stimulus design, and statistical methods for estimating connectivity using optogenetic stimulation. Since ref. [23], many studies have used GPs to infer the relationship between neural spiking and multi-dimensional covariates. This includes using GPs to infer place fields [24, 25] and orientation preference maps [26], as well as with Bayesian active learning techniques for inferring sensory receptive fields [27, 28]. However, none to our knowledge have used GP inference to model responses to optogenetic stimulation. Closed-loop methods for optimal control of neural activity using onephoton or electrical stimulation have been recently explored [29-34], though these approaches lack the necessary spatial specificity for highly precise ensemble control. Ref. [35] considers optimising spike-timing using two-photon stimulation (in addition to electrical stimulation), but focused on single neurons and did not use GP estimation methods. Finally, a number of papers have developed statistical models of optogenetic data to infer functional or synaptic connectivity [36-43], but do not consider identifying the exact stimulation parameters needed to evoke specific activity patterns. Here, we unify these three approaches to develop a novel computational framework for optimal holographic stimulation of neural ensembles.

3 Methods

3.1 Optogenetic receptive field model

Given a population of N neurons, we first wish to learn ORF functions g n that model how each neuron responds to optogenetic stimulation at different laser powers and locations near the cell soma. Ideally, ORF mapping should be completed quickly, stimulating as few times as possible, so that stimulus optimisation and the desired experiment can begin. Therefore, an ORF model should leverage prior knowledge of how neurons respond to stimulation. Importantly, spike probability should increase monotonically with laser power until saturation [42], and the ORF should approximately match the shape of the somatic membrane (though enlarged to account for the ∼ 10 µ m diameter of the

spike

no

response

Holographic

ensemble

stimulation

a

c

opsin concentration

low

high

Target neuron

Optimised holo

Naive holo

b

holographic disk [22]). However, any residual expression of opsin molecules in the proximal dendrites of a neuron creates unique differences in its ORF shape compared to other neurons (Figure 1c). Further, neurons can express opsin in variable concentrations and vary in their intrinsic excitability, leading to an unpredictable dependence of spiking on power. Thus, we need an approach to modelling an ORF that roughly describes the typical shape and photosensitivity of a cell, but that can flexibly adapt to variation across neurons.

To simultaneously account for these factors, we use a novel variant of the GP-Bernoulli model. Let y nt ∈ { 0 , 1 } denote the response of neuron n on trial t to a holographic stimulus x t ∈ R J × 3 . As our approach is designed for all-optical experiments, the response to stimulation is assumed to be observed for all neurons at once using calcium or voltage imaging. Each stimulus delivers two-photon excitation to J ≥ 1 different target locations, with x j t = ( c j 1 t , c j 2 t , I j t ) ∈ R 3 representing the two-dimensional coordinates and laser power of the j th target (though note that we can also handle three-dimensional hologram coordinates without any substantial changes to the model). While in some cases the targets will be far enough apart in space to not interact with each other, frequently a neuron will integrate two-photon excitation from multiple nearby targets, increasing the risk of OTS. To account for this, we model the probability of evoking a spike in neuron n by summing across all J points on that neuron's ORF,

where σ is the logistic sigmoid function and θ n ∈ R is a scalar threshold. We model each ORF using a three-dimensional GP prior that describes the effect of stimulating at a given location and power: g n ∼ GP ( m n ( · ) , k ( · , · )) , where m n and k are respectively the mean and covariance functions of the GP. During inference (as discussed below) we constrain g n to be non-negative, which ensures that stimulation cannot have an inhibitory effect. Further, using a non-negativity constraint rather than (e.g.) applying a rectifying nonlinearity to g n guarantees that the posterior distribution remains log-concave, facilitating tractable identification of the global optimum.

To encode prior knowledge about optogenetic stimulation, the mean function should have the property that the probability of evoking a spike increases with laser power but decays quickly as the hologram

is positioned further away from the nucleus. To this end, we set

for n = 1 , . . . , N (note that we suppress reference to the target dimension j for notational clarity here and in the definition of the covariance function below). Here c = ( c 1 , c 2 ) denotes the coordinates of the holographic target, L n is the location of neuron n , and ρ can be loosely interpreted as the average opsin expression level, determining how laser power affects the probability of spiking. While we treat ρ and σ 2 m as static hyperparameters, in future work one could consider a hierarchical model where the mean function is also learned, with its accuracy improving as additional ORFs are probed.

We model the covariance between points on the ORF using the radial basis function (RBF) kernel k , which regularises the ORF by encouraging it to vary smoothly through space and power. However, we note that other covariance kernels could be used here provided that they are differentiable with respect to x . We define

where Λ = diag ( λ s , λ s , λ I ) is a diagonal matrix with each diagonal term giving the characteristic lengthscale for the corresponding GP dimension. We assume that the lengthscales λ s in the two spatial dimensions are equal (a typical assumption in holography, see e.g. ref. [6]), but take a different lengthscale λ I for the laser power dimension. Finally, we denote the hyperparameters of the ORF prior by ϕ = ( α 2 , σ 2 d , λ s , λ I , ρ, σ 2 m ) .

3.2 Inference

During an ORF mapping experiment we first collect calibration data by performing stimulation in a series of trials x 1 , . . . , x T , where each x t targets J different locations and laser powers near a random set of neurons. Then we combine these stimuli with the corresponding observations of any evoked activity y : , 1 , . . . , y : ,T to estimate the ORFs. Here each y : ,t ∈ { 0 , 1 } N describes the population response to the stimulus x t . Due to the non-negativity constraints on g n , the posterior p ( g n | y n , { x t } T t =1 , θ n , ϕ ) is not itself a GP, and hence we perform maximum a posteriori (MAP) inference rather than attempting to obtain a complete description of posterior uncertainty (though see the supplementary material for a full treatment of posterior uncertainty in the J = 1 case). To this end, the MAP estimates are obtained by maximising the log-posterior, which, via Bayes rule, is equivalent to computing

Note that we have abbreviated ˆ g n ( X ) (the MAP version of the ORF) as ˆ g n . To solve Equation 4 we alternate between using Newton's method with a log-barrier to update g n (such that the optimised ORFs respect the non-negativity constraints), and performing gradient descent steps for updating θ n . We also adaptively set the Newton step-size using a standard backtracking line-search method.

3.3 Optimisation approach

Given the estimated ORFs, we then aim to identify holographic stimuli that evoke specific neural activity patterns as accurately as possible (i.e., minimising off-target activation). To do this, we introduce a novel optimisation approach that exploits properties of the GP to compute gradients of an error function between a target activity pattern and the (predicted) evoked activity. To this end, let G = { ˆ g n , ˆ θ n } N n =1 be the MAP-estimated ORFs, and define the predicted evoked activity as

where ˆ γ n ( x ) = ∑ J j =1 ˆ g n ( x j ) . Let Ω ∈ { 0 , 1 } N denote a target activity pattern. The optimal holographic stimulus is then

Note that we include I max as an upper bound on the laser power, which can be set to match the power practically deliverable by the microscopy system or to prevent tissue damage due to excess heating.

We propose to solve Equation 6 using a projected gradient descent algorithm. While for the ORF mapping phase we only need to compute the ORFs ˆ g n at a small set of stimulation points x t , for the optimisation phase we must evaluate the gradients of Equation 6 (and therefore of the ORFs) at a series of new, unobserved points constituting an optimisation path. However, it is not immediately obvious how to evaluate these gradients for a general nonparametric function g n . The key idea for our approach is that at each step, gradient descent will provide an updated estimate x ∗ of the optimal stimulus, which will act as a "test point" commonly used in GP regression. We then leverage properties of the GP to perform inference of the ORF gradients at x ∗ , and subsequently perform a gradient descent update that reduces the value of the objective function in Equation 6.

Inferring the ORF gradients relies on the fact that a GP and its derivative are jointly GP-distributed, and hence the derivative can be inferred just from a small number of observations of the GP itself. To this end, note that the covariance between a GP and its derivative is given by [45, Sec 9.4]

where d is any dimension of x ∗ and where we have suppressed reference to the hologram target j for notational clarity here and in the following equation. The expression in Equation 7 can be used to obtain a predictive distribution over derivative functions consistent with the observed data points, which we use to evaluate the derivatives of ˆ g n at the test point x ∗ via

where X ∈ R JT × 3 is the collection of JT unique points on the ORF that were probed during ORF mapping, g n ( X ) , m n ( X ) ∈ R JT give the value of the GP and mean function evaluated at each such point, and where K is the corresponding GP covariance matrix obtained by evaluating the covariance kernel k at every pair of elements of X . Further details are provided in the supplementary material.

Equation 8 allows us to define a closed-form gradient of the objective function, which we use in a projected gradient descent algorithm (Algorithm 1). Note, however, that the objective function is not convex in x , and therefore such updates are only guaranteed to converge to a locally optimal solution. We therefore typically run Algorithm 1 with several random initialisations and select the optimised target with the lowest predicted error.

Algorithm 1: Bayesian target optimisation (Bataro).

1 Compute MAP estimates of ORFs { ˆ g n , ˆ θ n } N n =1 from calibration data { y n } N n =1 , { x t } T t =1 using Newton's method with log-barrier.

2 Initialise targets x ∈ R J × 3 to random locations near the somas of the J target neurons and with random laser powers.

3 while target not converged do

4 Construct gradient vectors ∇ x ˆ γ n ( x ) for n = 1 , . . . , N using inference of ORF derivatives (Equation 8).

5 Set δ x = -2 ∑ N n =1 (Ω n -σ (ˆ γ n ( x ) -ˆ θ n )) σ ′ (ˆ γ n ( x ) -ˆ θ n ) ∇ x ˆ γ n ( x ) .

6 Perform gradient descent update x ← x + β δ x with step-size β .

7 Project laser power onto feasible domain, I j ← min ( I j , I max ) for j = 1 , . . . , J .

4 Results

4.1 Simulations

We first tested Bataro by simulating an optical "write-in" experiment, where we attempted to write specific activity patterns into a hypothetical neural population. To do this, we positioned 50 neurons randomly in a 250 µ m × 250 µ m field of view (FOV), sampled each neuron's ORF from its GP

a

b

c

d

random

6-target

ensembles

prior, and generated responses to ensemble stimulation. To quantify the optical write-in error, we used the sum of squared errors ∑ n (Ω n -y n ( x )) 2 , where y n ( x ) is the ground truth probability of neuron n spiking in response to stimulus x . This error takes the interpretable value of ∼ m if m non-target neurons are inadvertently recruited due to OTS. In this simulation, holographic stimulation of an example ensemble consisting of 6 selected neurons resulted in a substantial recruitment of activity from non-target neurons due to OTS, especially when multiple holograms converged on nearby non-target neurons (Figure 2a; optical write-in error = 7.84).

To optimally reposition the holographic targets, we probed the population's ORFs using 10-target ensemble stimulation, and then used Bataro (Algorithm 1). This successfully repositioned holographic targets an appropriate distance from the nuclei of the target neurons, such that while minimal excitation was applied to the non-target neurons, the target neurons were still driven to spike with high probability (Figure 2b,c; write-in error = 1.61). To confirm that the improvement in write-in precision was robust, we repeated the optimisation for 20 additional random 6-target ensembles and found that off-target effects were substantially reduced in every case, with the write-in error reducing by 75% on average compared to naively stimulating the nucleus of each neuron (Figure 2d).

Next, we investigated two variables known to critically constrain the ability to evoke specific neural activity patterns: (1) the density of opsin-expressing neurons in the stimulation FOV, and (2) the size of the ensemble that one is attempting to activate. Figure 3a illustrates how increasing the number of opsin-expressing neurons introduces many more potential non-target neurons that could be inadvertently activated. This causes errors induced by direct nuclear stimulation of (e.g.) 5-neuron ensembles to grow rapidly (Figure 3b, grey curve), though note that the magnitude of the error depends on many experimental factors including the size of the ORF, numerical aperture of the microscope, and the soma-targeting efficacy. By optimising holographic target locations and laser powers, the average write-in error was reduced by 76% relative to nuclear stimulation (Figure 3b, red curve). Similarly, in these simulations, nuclear stimulation resulted in an error that grew almost

a

50

100

150

Population size

0

2

4

6

8

10

12

14

16

18

Write-in error

b

Nuclear

Optimised

GLYPH<c=20,font=/ZRNEDJ+Helvetica>

GLYPH<c=24,font=/ZRNEDJ+Helvetica>

GLYPH<c=20,font=/ZRNEDJ+Helvetica>GLYPH<c=19,font=/ZRNEDJ+Helvetica>

GLYPH<c=20,font=/ZRNEDJ+Helvetica>GLYPH<c=24,font=/ZRNEDJ+Helvetica>

6WLPXODWHG HQVHPEOH VL]H

GLYPH<c=19,font=/ZRNEDJ+Helvetica>

GLYPH<c=21,font=/ZRNEDJ+Helvetica>

GLYPH<c=23,font=/ZRNEDJ+Helvetica>

GLYPH<c=25,font=/ZRNEDJ+Helvetica>

GLYPH<c=27,font=/ZRNEDJ+Helvetica>

GLYPH<c=20,font=/ZRNEDJ+Helvetica>GLYPH<c=19,font=/ZRNEDJ+Helvetica>

GLYPH<c=20,font=/ZRNEDJ+Helvetica>GLYPH<c=21,font=/ZRNEDJ+Helvetica>

GLYPH<c=20,font=/ZRNEDJ+Helvetica>GLYPH<c=23,font=/ZRNEDJ+Helvetica>

GLYPH<c=20,font=/ZRNEDJ+Helvetica>GLYPH<c=25,font=/ZRNEDJ+Helvetica>

GLYPH<c=20,font=/ZRNEDJ+Helvetica>GLYPH<c=27,font=/ZRNEDJ+Helvetica>

:ULWHGLYPH<c=16,font=/ZRNEDJ+Helvetica>LQ HUURU

c

a

b

c

30 μm

00

+

00

?

?

3

Q

?

?

?

?

?

G

G

8.

Q.

00

C

?

?

G

Q

Q

C

?

?

2

?

?

G

?

?

2G

?

?

?

Y

00

100

200

D

300

?

400

directly proportionally to the size of the stimulated ensemble, indicating that almost as many nontarget neurons were activated as target neurons. However, Bataro reduced the average write-in error by 69% across ensembles of size 1-15 (Figure 3c).

4.2 Application to holographic optogenetics experiments

To test whether Bataro could eliminate OTS in realistic settings, we created synthetic optogenetics experiments involving > 100 neurons from a small number ( n =4) of detailed cell-attached recordings in slice [42]. Briefly, each slice experiment was performed by first establishing a cell-attached electrophysiological recording of a single L2/3 interneuron from mouse V1 expressing the somatargeted opsin ChroME2f. Then, the recorded neuron was optogenetically stimulated at a dense grid

target

neuron

Δxy = 2 μm

Δz = 11 μm

target

b

deeper

a

75 μm

50 μm

target

neuron

25 μm

0 μm

Δxy = 14 μm

Δz = 1 μm

target

f

g

c

75 μm

50 μm

25 μm

0 μm

e

d

20 μm

z-projection

?

?

Q

?

?

?

Q

of locations surrounding the cell and at a range of laser powers to comprehensively map its ORF (see Figure 1c, "100% sampled" column, also see supplementary material for additional examples). We used this data to create a set of four "ground truth" ORFs by fitting the GP-Bernoulli model from Equation 1, where the hyperparameters of the GP covariance kernel were selected using the crossvalidated predictive log-likelihood. Next, we used a fluorescence image from a separate experiment to extract the locations of 116 putative opsin-expressing neurons, and randomly assigned each putative neuron one of four ground-truth ORFs. Together, this enabled us to simulate responses to optogenetic stimulation at arbitrary locations and laser powers and for a large number of neurons that were realistically distributed in space.

We sampled responses to optogenetic stimulation at a sparse grid of locations surrounding each neuron to map their ORFs in a hypothetical experiment (see supplementary methods for details). Then, we used Bataro to compute single-target stimuli that activated each neuron individually and minimised OTS (Figure 4a). We found that, on the one hand, a subset of neurons required no change from nuclear stimulation at high power due to being spatially isolated. However, regions with large numbers of closely-packed, opsin-expressing neurons required repositioning of the holographic targets (Figure 4a, right) and adjustment to the laser powers (Figure 4b) to eliminate OTS (average distance between nuclear and optimised stimuli, 4.9 µ m). Across all single-target holograms, target optimisation reduced the average write-in error by 85% (Figure 4c). We also performed a control

analysis by matching the nuclear stimulation laser power to the average optimised laser power and obtained similar results (Figure S3).

Some holographic microscopy systems cannot position holograms at arbitrary continuous depths, instead either requiring holograms to be positioned in two dimensions or on a preselected number of discrete stimulation planes. Hence, thus far, we focused our efforts on the much more common case of two-dimensional optogenetics experiments. However, OTS is generally observed to be even more prevalent in three dimensions, as the optical point spread function is elongated in the "z"-axis [15, 6, 13, 17]. We therefore tested Bataro in the more general setting of three spatial dimensions and laser power (i.e., four dimensions in total), under the assumption that some microscopy systems may be able to flexibly reposition holograms continuously across depths.

We used the fact that the cell-attached recordings were repeated at multiple depths to generate synthetic optogenetics experiments with four-dimensional ORFs (Figure 5a,b). Optimisation of holographic targets successfully reduced OTS by exploiting both the x/y dimensions (Figure 5a), as well the z dimension (Figure 5b), in addition to automatically adjusting the laser power where needed. Repeating the optimisation for every single-target hologram reduced the average optical write-in error by 89% (from 0.7 to 0.08, Figure 5c-g; average distance between nuclear and optimised targets, 8.3 µ m), demonstrating the feasibility of largely eliminating OTS in optogenetics experiments involving all three spatial dimensions.

5 Conclusions

We developed Bataro, a novel computational framework for two-photon optogenetics that optimises holographic stimuli to evoke neural activity patterns with minimal off-target activation. Our preliminary in vitro experiments predict that specific adjustments to holographic target placements of 5-10 µ m (on average) could substantially reduce OTS, even in the more challenging regime of three-dimensional optogenetic stimulation. Our key idea was to simultaneously exploit variability in photoexcitability, residual opsin expression in proximal dendrites (due to imperfections in opsin soma-targeting), and the specific spatial arrangement of neurons in the stimulation FOV. Optimisation using these three factors ultimately led to reductions in the optical write-in error of 85-89% for single-target holograms using data from in vitro two-photon optogenetics experiments.

Whether achieving such stimulus precision is worth spending valuable experiment time mapping ORFs is, of course, highly dependent on the application. While for some experiments minimising off-target activation may not be critical, there are many cases where stimulating with as close to "true" single-cell precision as possible is essential to testing a scientific hypothesis. For example, "hub" neurons in the hippocampus are believed to orchestrate network-level function through their unilateral activity [46, 47]. Thus, using two-photon optogenetics to test whether an individual neuron represents a functional hub by stimulating that neuron alone requires true single-cell precision. In such cases, we believe devoting experiment time to ORF mapping and stimulus optimisation is a necessary trade-off.

Throughout our analysis we have assumed that ORFs are stable over time (i.e., are stationary). However, desensitisation of opsins following prolonged illumination has previously been observed [11], indicating that some nonstationarity could be encountered in real experiments. To account for this, we note that after the ORFs have been mapped and holographic stimuli optimised, it is straightforward to periodically recompute the MAP estimate of the ORF as the experiment proceeds. This would provide an updated estimate of the laser power required to maintain optimal precision. We hypothesise that this would not require repeating the entire ORF mapping phase as we do not expect the shape of the ORF to change drastically beyond desensitisation.

Standard GP regression methods scale cubically with the number of locations and laser powers probed, and therefore estimation of each neuron's ORF could encroach on experiment time if ORFs are mapped at high detail just due to required computation time. However, computational efficiency could be improved by appealing to modern GP techniques such as inducing point methods [48], or by adaptively mapping ORFs using Bayesian active learning to minimise the number of probed locations and powers [49, 50]. While in this work we have focused primarily on demonstrating the feasibility of overcoming OTS, in future work we plan to minimise both required experiment and computation time in order to deploy Bataro online.

References

John Peter Rickgauer and David W Tank. Two-photon excitation of channelrhodopsin-2 at saturation. Proceedings of the National Academy of Sciences , 106(35):15025-15030, 2009.

Eirini Papagiakoumou, Francesca Anselmi, Aurélien Bègue, Vincent De Sars, Jesper Glückstad, Ehud Y Isacoff, and Valentina Emiliani. Scanless two-photon excitation of channelrhodopsin-2. Nature Methods , 7(10):848-854, 2010.

Adam M Packer, Darcy S Peterka, Jan J Hirtz, Rohit Prakash, Karl Deisseroth, and Rafael Yuste. Two-photon optogenetics of dendritic spines and neural circuits. Nature Methods , 9(12):1202-1205, 2012.

Rohit Prakash, Ofer Yizhar, Benjamin Grewe, Charu Ramakrishnan, Nancy Wang, Inbal Goshen, Adam M Packer, Darcy S Peterka, Rafael Yuste, Mark J Schnitzer, et al. Two-photon optogenetic toolbox for fast inhibition, excitation and bistable modulation. Nature Methods , 9(12):1171-1179, 2012.

Oscar Hernandez, Eirini Papagiakoumou, Dimitrii Tanese, Kevin Fidelin, Claire Wyart, and Valentina Emiliani. Three-dimensional spatiotemporal focusing of holographic patterns. Nature Communications , 7(1):1-11, 2016.

Nicolas C Pégard, Alan R Mardinly, Ian Antón Oldenburg, Savitha Sridharan, Laura Waller, and Hillel Adesnik. Three-dimensional scanless holographic optogenetics with temporal focusing (3d-shot). Nature Communications , 8(1):1-14, 2017.

Emiliano Ronzitti, Cathie Ventalon, Marco Canepari, Benoît C Forget, Eirini Papagiakoumou, and Valentina Emiliani. Recent advances in patterned photostimulation for optogenetics. Journal of Optics , 19(11):113001, 2017.

Nicolò Accanto, Clément Molinier, Dimitrii Tanese, Emiliano Ronzitti, Zachary L Newman, Claire Wyart, Ehud Isacoff, Eirini Papagiakoumou, and Valentina Emiliani. Multiplexed temporally focused light shaping for high-resolution multi-cell targeting. Optica , 5(11):14781491, 2018.

Weijian Yang, Luis Carrillo-Reid, Yuki Bando, Darcy S Peterka, and Rafael Yuste. Simultaneous two-photon imaging and two-photon optogenetics of cortical circuits in three dimensions. elife , 7:e32671, 2018.

Alan R Mardinly, Ian Antón Oldenburg, Nicolas C Pégard, Savitha Sridharan, Evan H Lyall, Kirill Chesnov, Stephen G Brohawn, Laura Waller, and Hillel Adesnik. Precise multimodal optical control of neural ensemble activity. Nature Neuroscience , 21(6):881-893, 2018.

James H Marshel, Yoon Seok Kim, Timothy A Machado, Sean Quirin, Brandon Benson, Jonathan Kadmon, Cephra Raja, Adelaida Chibukhchyan, Charu Ramakrishnan, Masatoshi Inoue, et al. Cortical layer-specific critical dynamics triggering perception. Science , 365(6453):eaaw5202, 2019.

Luis Carrillo-Reid, Shuting Han, Weijian Yang, Alejandro Akrouh, and Rafael Yuste. Controlling visually guided behavior by holographic recalling of cortical ensembles. Cell , 178(2):447457, 2019.

Hillel Adesnik and Lamiae Abdeladim. Probing neural codes with two-photon holographic optogenetics. Nature Neuroscience , 24(10):1356-1366, 2021.

Eirini Papagiakoumou, Emiliano Ronzitti, and Valentina Emiliani. Scanless two-photon excitation with temporal focusing. Nature Methods , 17(6):571-581, 2020.

Christopher A Baker, Yishai M Elyada, Andres Parra, and M McLean Bolton. Cellular resolution circuit mapping with temporal-focused excitation of soma-targeted channelrhodopsin. Elife , 5:e14193, 2016.

Or A Shemesh, Dimitrii Tanese, Valeria Zampini, Changyang Linghu, Kiryl Piatkevich, Emiliano Ronzitti, Eirini Papagiakoumou, Edward S Boyden, and Valentina Emiliani. Temporally precise single-cell-resolution optogenetics. Nature Neuroscience , 20(12):1796-1806, 2017.

Travis A Hage, Alice Bosma-Moody, Christopher A Baker, Megan B Kratz, Luke Campagnola, Tim Jarsky, Hongkui Zeng, and Gabe J Murphy. Synaptic connectivity to l2/3 of primary visual cortex measured by two-photon optogenetic stimulation. Elife , 11:e71103, 2022.

Valentina Emiliani, Adam E Cohen, Karl Deisseroth, and Michael Häusser. All-optical interrogation of neural circuits. Journal of Neuroscience , 35(41):13917-13926, 2015.

Adam M Packer, Lloyd E Russell, Henry WP Dalgleish, and Michael Häusser. Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo. Nature Methods , 12(2):140-146, 2015.

Zihui Zhang, Lloyd E Russell, Adam M Packer, Oliver M Gauld, and Michael Häusser. Closedloop all-optical interrogation of neural circuits in vivo. Nature Methods , 15(12):1037-1040, 2018.

Lloyd E Russell, Henry WP Dalgleish, Rebecca Nutbrown, Oliver M Gauld, Dustin Herrmann, Mehmet Fi¸ sek, Adam M Packer, and Michael Häusser. All-optical interrogation of neural circuits in behaving mice. Nature Protocols , 17(7):1579-1620, 2022.

Hayley A Bounds, Masato Sadahiro, William D Hendricks, Marta Gajowa, Ian Antón Oldenburg, Karthika Gopakumar, Daniel Quintana, Tanya Daigle, Hongkui Zeng, and Hillel Adesnik. Multifunctional cre-dependent transgenic mice for high-precision all-optical interrogation of neural circuits. bioRxiv , page 463223, 2021.

Kamiar Rahnama Rad and Liam Paninski. Efficient, adaptive estimation of two-dimensional firing rate surfaces via gaussian process methods. Network: Computation in Neural Systems , 21(3-4):142-168, 2010.

Cristina Savin and Gasper Tkacik. Estimating nonlinear neural response functions using gp priors and kronecker methods. Advances in Neural Information Processing Systems , 29, 2016.

M Rule, P Chaudhuri-Vayalambrone, Marino Krstulovic, Marius Bauza, Julija Krupic, and Timothy O'Leary. Variational log-gaussian point-process methods for grid cells. bioRxiv , pages 2023-03, 2023.

Jakob H Macke, Sebastian Gerwinn, Leonard E White, Matthias Kaschube, and Matthias Bethge. Gaussian process methods for estimating cortical maps. NeuroImage , 56(2):570-581, 2011.

Mijung Park, Greg Horwitz, and Jonathan Pillow. Active learning of neural response functions with gaussian processes. Advances in Neural Information Processing Systems , 24, 2011.

Mijung Park, J Patrick Weller, Gregory D Horwitz, and Jonathan W Pillow. Bayesian active learning of neural firing rate maps with transformed gaussian process priors. Neural Computation , 26(8):1519-1541, 2014.

Jonathan P Newman, Ming-fai Fong, Daniel C Millard, Clarissa J Whitmire, Garrett B Stanley, and Steve M Potter. Optogenetic feedback control of neural activity. Elife , 4:e07192, 2015.

Yuxiao Yang, Allison T Connolly, and Maryam M Shanechi. A control-theoretic system identification framework and a real-time closed-loop clinical simulation testbed for electrical brain stimulation. Journal of Neural Engineering , 15(6):066007, 2018.

Michael F Bolus, Adam A Willats, Clarissa J Whitmire, Christopher J Rozell, and Garrett B Stanley. Design strategies for dynamic closed-loop optogenetic neurocontrol in vivo. Journal of Neural Engineering , 15(2):026011, 2018.

Nishal Shah, Sasidhar Madugula, Pawel Hottowy, Alexander Sher, Alan Litke, Liam Paninski, and EJ Chichilnisky. Efficient characterization of electrically evoked responses for neural interfaces. Advances in Neural Information Processing Systems , 32, 2019.

Michael F Bolus, Adam A Willats, Christopher J Rozell, and Garrett B Stanley. State-space optimal feedback control of optogenetically driven neural activity. Journal of Neural Engineering , 18(3):036006, 2021.

Yuxiao Yang, Shaoyu Qiao, Omid G Sani, J Isaac Sedillo, Breonna Ferrentino, Bijan Pesaran, and Maryam M Shanechi. Modelling and prediction of the dynamic responses of large-scale brain networks during direct electrical stimulation. Nature Biomedical Engineering , 5(4):324345, 2021.

Yashar Ahmadian, Adam M Packer, Rafael Yuste, and Liam Paninski. Designing optimal stimuli to control neuronal spike timing. Journal of Neurophysiology , 106(2):1038-1053, 2011.

Tao Hu and Dmitri Chklovskii. Reconstruction of sparse circuits using multi-neuronal excitation (rescume). Advances in Neural Information Processing Systems , 22:790-798, 2009.

Alyson K Fletcher, Sundeep Rangan, Lav R Varshney, and Aniruddha Bhargava. Neural reconstruction with approximate message passing (neuramp). Advances in Neural Information Processing Systems , 2011.

Ben Shababo, Brooks Paige, Ari Pakman, and Liam Paninski. Bayesian inference and online experimental design for mapping neural microcircuits. Advances in Neural Information Processing Systems , 26:1304-1312, 2013.

Alyson K Fletcher and Sundeep Rangan. Scalable inference for neuronal connectivity from calcium imaging. Advances in Neural Information Processing Systems , 2014.

Laurence Aitchison, Lloyd Russell, Adam Packer, Jinyao Yan, Philippe Castonguay, Michael Häusser, and Srinivas C Turaga. Model-based bayesian inference of neural activity and connectivity from all-optical interrogation of a neural circuit. Advances in Neural Information Processing Systems , 2017.

Anne Draelos and John Pearson. Online neural connectivity estimation with noisy group testing. Advances in Neural Information Processing Systems , 33, 2020.

Marcus A Triplett, Marta Gajowa, Benjamin Antin, Masato Sadahiro, Hillel Adesnik, and Liam Paninski. Rapid learning of neural circuitry from holographic ensemble stimulation enabled by model-based compressed sensing. bioRxiv , page 507926, 2022.

Yoav Printz, Pritish Patil, Mathias Mahn, Asaf Benjamin, Anna Litvin, Rivka Levy, Max Bringmann, and Ofer Yizhar. Determinants of functional synaptic connectivity among amygdalaprojecting prefrontal cortical neurons in male mice. Nature Communications , 14(1):1667, 2023.

Savitha Sridharan, Marta A Gajowa, Mora B Ogando, Uday K Jagadisan, Lamiae Abdeladim, Masato Sadahiro, Hayley A Bounds, William D Hendricks, Toby S Turney, Ian Tayler, et al. High-performance microbial opsins for spatially and temporally precise perturbations of large neuronal networks. Neuron , 2022.

Christopher KI Williams and Carl Edward Rasmussen. Gaussian Processes for Machine Learning , volume 2. MIT Press Cambridge, MA, 2006.

Paolo Bonifazi, Miri Goldin, Michel A Picardo, Isabel Jorquera, A Cattani, Gregory Bianconi, Alfonso Represa, Yehezkel Ben-Ari, and Rosa Cossart. Gabaergic hub neurons orchestrate synchrony in developing hippocampal networks. Science , 326(5958):1419-1424, 2009.

Marco Bocchio, Claire Gouny, David Angulo-Garcia, Tom Toulat, Thomas Tressard, Eleonora Quiroli, Agnès Baude, and Rosa Cossart. Hippocampal hub neurons maintain distinct connectivity throughout their lifetime. Nature Communications , 11(1):4559, 2020.

Michalis Titsias. Variational learning of inducing variables in sparse gaussian processes. In Artificial Intelligence and Statistics (AISTATS) , pages 567-574. PMLR, 2009.

Jeremy Lewi, Robert Butera, and Liam Paninski. Sequential optimal design of neurophysiology experiments. Neural Computation , 21(3):619-687, 2009.

Jonathan W Pillow and Mijung Park. Adaptive bayesian methods for closed-loop neurophysiology. Closed Loop Neuroscience , pages 3-18, 2016.

Andrew James McHutchon et al. Nonlinear modelling and control using Gaussian processes . PhD thesis, Citeseer, 2015.

6 Supplementary material

6.1 Animal ethics statement

All experiments on animals were conducted with approval of the Animal Care and Use Committee of the University of California, Berkeley.

6.2 Compute

All computational procedures were performed either on a desktop workstation running Ubuntu 18.04 with an Intel Xeon E5-2620 v4 CPU, four GTX 1080 Ti GPUs, and 112GB RAM, or on the Axon computer cluster based at the Zuckerman Institute (Columbia University) using nodes comprised of two Xeon E5-2660 v4 CPUs, eight GTX 1080 Ti GPUs, and 125GB RAM.

6.3 Broader societal impact

Our work is significant for interventional approaches to studying the brain and its connection to disease. By minimising off-target activation, Bayesian target optimisation could enable (e.g.) more precise synaptic connectivity mapping, improving our understanding of neural circuitry. This advancement has potential implications for understanding brain disorders like epilepsy, where abnormal synaptic connections are central to seizure generation and propagation. Deepening our understanding of these diseases can lead to enhanced targeted interventions and more effective therapeutic strategies, benefiting individuals with neurological disorders.

6.4 Code availability

An open-source implementation of Bayesian target optimisation is available in Python at https: //github.com/marcustriplett/bataro .

6.5 Single-target holographic stimulus optimisation with posterior uncertainty

Here we provide further mathematical details for optimising holographic stimuli. First, we develop the approach for single optogenetic targets, as this is most closely related to existing GP-based receptive field inference techniques. The single-target case also allows us to have a full treatment of posterior uncertainty (unlike for optimising ensemble stimuli) which may be desired in certain applications.

Optogenetic receptive field model. We use a GP-Bernoulli approach to model the response y nt of neuron n on trial t to a single-target stimulus x t ,

where the stimulus x t = ( c 1 t , c 2 t , I t ) ∈ R 3 represents the two-dimensional coordinates and laser power of the t -th hologram. Each ORF follows a three-dimensional GP prior g n ∼ GP ( m n ( · ) , k ( · , · )) , where m n and k again are the mean and covariance functions of the GP.

Posterior inference. Unlike for ensemble stimulation, for single-target stimulation we do not require that the ORF g n is non-negative. This is because now if a point on the ORF becomes inhibitory (by taking a negative value), it will not conflict with excitation from any other hologram. Consequently, the posterior of g n is a GP, which allows us to work with a full description of posterior uncertainty. To compute the posterior, we use the conventional Laplace approximation. Briefly, this consists of approximating the posterior using a multivariate normal q ( g n | µ n , Σ n ) = Normal ( g n | µ n , Σ n ) ≈ p ( g n | y n , X , ϕ ) . The mean µ n is obtained by maximising the log-posterior, given by the expression

where X = ( x 1 , . . . , x T ) and where const does not depend on g n . Since the posterior is log-concave in g n , we use Newton's method to identify the global optimum of Equation 10, and adaptively set the Newton step-size using a standard backtracking line-search method. Letting H = ∇∇ g n ln p ( g n |

y n , X , ϕ ) be the Hessian of the log-posterior, the posterior covariance matrix is obtained by setting Σ n = -H -1 | g n = µ n .

Target optimisation . Let G = ( g 1 , . . . , g N ) , and define the predicted evoked activity for single holographic targets as ˆ y ( x , G ) = ( σ ( g 1 ( x )) , . . . , σ ( g N ( x ))) . To minimise the error between a target binary activity pattern Ω ∈ { 0 , 1 } N and the predicted evoked activity, we solve an optimisation problem that accounts for the uncertainty in the ORF estimates:

where q ( G | µ , Σ ) = ∏ N n =1 q ( g n | µ n , Σ n ) gives the joint posterior across all ORFs. To solve Equation 11, we first sample ORFs g ( s ) n (for s = 1 , . . . , S ) from their posterior distributions to approximate the expected error at the current estimate x ∗ ,

Then, we compute the partial derivative (in dimension d ) of the expected error by differentiating through the Monte Carlo approximation,

Next we must evaluate the partial derivative on the right-hand side of Equation 13. We use the fact that a GP and its derivative are jointly GP-distributed, and hence infer the derivative from observations of the ORF. The covariance between a GP and its derivative is given by [45, Sec 9.4]

where the second equality is specific to the RBF covariance. Thus, we can use Equation 14 to obtain the posterior predictive mean for the derivative GPs in closed form as [51, Sec 2.7]

Here X = ( x 1 , . . . , x T ) is the collection of unique points on the ORF probed during calibration. If Equation 15 is combined with an expression for the posterior predictive variance, one obtains a full predictive distribution over derivative functions consistent with the observed neural responses. However, rather than working with this full distribution, we instead use Equation 15 to approximate the derivatives of the Monte Carlo samples by replacing the posterior mean µ n with a Monte Carlo sample,

Equation 16 then allows us to define a closed-form approximate gradient ˜ ∇ x ∗ g ( s ) n at test point x ∗ , defined as

which we use in the single-target projected gradient descent algorithm (Algorithm 2). Note that one could also consider a quadrature approach to solving Equation 12, which may be more efficient than Monte Carlo sampling. However, the presentation of the Monte Carlo approach is instructive for deriving the optimisation of ensemble stimuli below.

Algorithm 2: Projected Monte Carlo gradient descent algorithm for optimising single-target holograms

1 Infer ORF posterior q ( G | µ , Σ ) from calibration data { y n } N n =1 , X using the Laplace approximation. 2 Precompute the negative of the Hessian W n = -∇∇ ln p ( y n | X , g n ) | g n = µ n for each n . 3 Initialise x to random location near soma of target neuron and with random laser power. 4 while target not converged do 5 for n = 1 , . . . , N do 6 Compute mean and variance of posterior predictive distribution at current target estimate x via µ n ( x ) = m n ( x ) + k ( X , x ) ⊤ K -1 ( µ n -m n ( X )) , and σ 2 n ( x ) = k ( x , x ) -k ( X , x ) ⊤ ( K + W -1 n ) -1 k ( X , x ) . 7 Sample ORFs at the current target estimate, g ( s ) n ( x ) ∼ Normal ( µ n ( x ) , σ 2 n ( x )) for s = 1 , . . . , S . 8 Construct approximate gradients ˜ ∇ x g ( s ) n for s = 1 , . . . , S using Equation 17. 9 end 10 Set δ x = -2 S ∑ S s =1 ∑ N n =1 (Ω n -σ ( g ( s ) n ( x )) σ ′ ( g ( s ) n ( x )) ˜ ∇ x g ( s ) n ( x ) as per Equation 13. 11 Perform gradient descent update, x ← x + β δ x with step-size β . 12 Project laser power onto feasible domain, I ← min ( I, I max ) . 13 end

6.6 Additional details on ensemble stimulus optimisation approach

The approach for optimising holographic ensemble stimuli is based on the approach for single-target optimisation, but modified to account for differences in the ORF model and inference. In particular, we again seek to minimise the error between a target activity pattern Ω and the predicted evoked activity, but now using the MAP estimates G = { ˆ g n , ˆ θ n } N n =1 in place of the full posterior distributions. Let ˆ y ( x , G ) = ( σ (ˆ γ 1 ( x ) -ˆ θ 1 ) , . . . , σ (ˆ γ N ( x ) -ˆ θ N )) be the predicted population response to an ensemble stimulus, where ˆ γ n ( x ) = ∑ J j =1 ˆ g n ( x j ) . The optimal ensemble stimulus is now

such that 0 ≤ I ≤ I max. Evaluating the partial derivative of Equation 18 with respect to dimension d of a test point x ∗ yields,

The derivative on the right-hand side of Equation 19 is given by ∂ ∂x ∗ d ˆ γ n ( x ) = ∑ J j =1 ∂ ∂x ∗ d ˆ g n ( x j ) , which requires computing the derivative of ˆ g n ( x j ) . To evaluate this derivative, we use a similar trick to Equation 16, but substituting the MAP estimate in place of the posterior mean or Monte Carlo sample,

This expression can also be arrived at by first evaluating the posterior predictive mean of g n ( x ∗ ) , and then differentiating with respect to x ∗ d .

We use Equation 20 to define a closed-form gradient ∇ x ∗ ˆ γ n at test point x ∗ via

Finally, Equation 21 is used in the projected gradient descent algorithm for optimising ensemble stimuli (Algorithm 1).

1

5

10

15

Stimulated ensemble size

0

2

4

6

8

10

12

14

16

18

Write-in error

Nuclear

High coverage

Low coverage

-20

-10

0

10

20

X distance (

μ

m)

-20

-10

0

10

20

Y distance (

μ

m)

High coverage

-20

-10

0

10

20

X distance (

μ

m)

-20

-10

0

10

20

Y distance (

μ

m)

Low coverage

a

b

c

6.7 Further details on simulations and "synthetic" optogenetics experiments

Simulations consisted of both ORF mapping and stimulus optimisation phases. ORF mapping required probing responses to stimulation at a range of laser powers and stimulus locations. We defined a grid of stimulation points surrounding each neuron. In the spatial dimensions, the grid ranged from -20 µ mto 20 µ mrelative to the centroid of the neuron in steps of 10 µ m, and powers ranged from 30 mW to 70 mW in steps of 20 mW. The complete grid was thus given by the Cartesian product {-20 , -10 , 0 , 10 , 20 }×{-20 , -10 , 0 , 10 , 20 }×{ 30 , 50 , 70 } . For opsin-expressing neurons that were spaced far apart, this coarse-resolution grid was sufficient because risk of OTS was low, and therefore ORF mapping was not needed at high detail. On the other hand, as the density of opsin-expressing neurons increased, the grids surrounding each neuron increasingly overlapped with each other, resulting in much denser sampling of the ORFs.

For the synthetic optogenetics experiments (based on the cell-attached recordings), we used the same spatial grid spacing but used laser powers of 10, 25, and 40 mW to match the range of powers used in the underlying slice experiment, though note that the slice experiment had a denser spacing than our chosen 15 mW (see example loose-patch recordings below), which we chose to reduce the ORF mapping time. For the optogenetics experiments involving three spatial dimensions, we extended the grid sampling to include depths of -60 µ mto 60 µ min steps of 30 µ m. We also explored the effect of reducing the number of probed grid points to further reduce the time spent mapping ORFs, and found that Bayesian target optimisation maintained high performance when probing with a 3 × 3 spatial grid of {-12 , 0 , 12 } × {-12 , 0 , 12 } (Figure S1).

We selected the parameters of the GP covariance kernel using 5-fold cross-validation on a separate set of recordings that were made on the same set of four cells, ensuring the hyperparameter selection was using out-of-sample data. Cross-validation was performed using a grid search over a set of possible hyperparameters: the possible radial lengthscales were 2, 4, 8, 16, the power lengthscales were 2, 4, 8, 16, and the amplitudes were 1, 2, 4, 8, 16. For each hyperparameter combination θ and for each cell, we used Newton's method to fit the GP-Bernoulli model to 80% of the trials in the loosepatch data, yielding an ORF estimate ˆ g θ . On the remaining 20% of the trials (denoted as T held-out ), we evaluated the log-likelihood, ∑ t ∈T held-out { y t ln( σ (ˆ g θ ( x t ))) + (1 -y t ) ln(1 -σ (ˆ g θ ( x t ))) } . We averaged the log-likelihood across all five folds and across all four cells, and chose the hyperparameter combination θ that yielded the largest average log-likelihood, resulting in a radial lengthscale of 8, a power lengthscale of 16, and a kernel amplitude of 8.

The GP parameters for generating the simulations in Figure 3, inferring the resulting ORFs, and generating synthetic optogenetics experiments with two and three spatial dimensions are given in Table S1. For reference, a typical ORF mean function is given in Figure S2.

25

0

25

GP mean

70 mW

Sample 1

Sample 2

Sample 3

Sample 4

25

0

25

Y distance (

m)

50 mW

25

0

25

25

0

25

30 mW

25

0

25

25

0

25

X distance (

m)

25

0

25

25

0

25

0.0

0.5

1.0

Spike prob

Nuclear

Optimised

0.0

0.3

0.6

0.9

1.2

1.5

Write-in error

Nuclear

Optimised

(average)

30

32

34

36

38

40

Laser power (mW)

Average performance

(5 random repeats, matched powers)

6.8 Additional examples of optogenetic receptive fields from cell-attached recordings

Figures S4 to S7 show examples of four ORFs that have been comprehensively mapped using twophoton optogenetic stimulation and cell-attached recordings of evoked spikes. Note the unpredictable differences in ORF shape across laser powers and depths, motivating a nonparametric approach.

25

0

25

Depth = 75  m

Y distance (

m)

10 mW

15 mW

20 mW

30 mW

40 mW

25

0

25

50  m

Y distance (

m)

25

0

25

25  m

Y distance (

m)

25

0

25

0  m

Y distance (

m)

25

0

25

-25  m

Y distance (

m)

25

0

25

-50  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

-75  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

O

000000

ooo

oo

D

00

O

500000

000

o

O

O

oo

oo

oo

o

1

oo

o00000oo

oo0000000000

O

O

o0

oo

00000

O

00

Q

Q

Q

000000000

oo

oo

o0000

000000

oo

oo

Q

oooooo

oo000000000o

o00000000000

oo

oo

oo

O

oo

oo

oo

O

O

jooooo

000000

000000000000

0000000000o0

000000000000

oooo

Q

Q

Q

O

oooooo

C

000000000000

oo

Q

O

O

ooo

oo

QQ

O

O

Q

ooo

Q：

O

Q

Q

O

Q

0oooo

00000

oo

O

O

oo

ooo

Q

oo

25

0

25

Depth = 75  m

Y distance (

m)

10 mW

15 mW

20 mW

30 mW

40 mW

25

0

25

50  m

Y distance (

m)

25

0

25

25  m

Y distance (

m)

25

0

25

0  m

Y distance (

m)

25

0

25

-25  m

Y distance (

m)

25

0

25

-50  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

-75  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

1

O

Q

O

O

oo

o

oooooo

oo

00

oo

oo

O

00000

o0

oo

o00000000000

000000000000

o

O

o

0

oo

O

o

o

00：0

o

O

oo

oo

oo

o

0000000

0000

C

1

oooooo

O

O

Q

Q

Q

Q

Q

Q

oo

o0

Q

oo

oo

000000000000

O

000

000

000

000

000000000

C

000

C

o

oooooo

oooooo

oooo

O

C

00000000000o

oooo

00

ooo

oo

oo

oo

o0

a

000

oo

o

ooo

00000000000o

25

0

25

Depth = 75  m

Y distance (

m)

10 mW

15 mW

20 mW

30 mW

40 mW

25

0

25

50  m

Y distance (

m)

25

0

25

25  m

Y distance (

m)

25

0

25

0  m

Y distance (

m)

25

0

25

-25  m

Y distance (

m)

25

0

25

-50  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

-75  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

0000

Q

000

oooo

0000000000oo

oo

oo

Q

Q

Q

oooo

QQ

oo

O

o

00

0000

000

o00

o

oo

oo

O

o0

oo

oo

00000

0000000

00000

000000000000

1000000000000

00000000

O

oooooo

000000000000

000000000000

000000000000

O

000000000000

000000000000

oooooo

o0o

oo

00

0000000000o0

0o

oo

O

oo

oooo

o

000000000000

ooo

000000000000

o00000000000

o0o

000000000000

O

O

000000000000

00

00000o

00

Q

oo

O

0

00000

O

0000000

Q

Q

oo

O

Q

oo

oo

oooooo

000000000000

00000000o

o0

oo

000000000000

ooo

000000000000

25

0

25

Depth = 75  m

Y distance (

m)

10 mW

15 mW

20 mW

30 mW

40 mW

25

0

25

50  m

Y distance (

m)

25

0

25

25  m

Y distance (

m)

25

0

25

0  m

Y distance (

m)

25

0

25

-25  m

Y distance (

m)

25

0

25

-50  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

-75  m

Y distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

25

0

25

X distance (

m)

O

00000000000o

000000000000

O

Q

O

Q

oooooooo

Q

Q

Q

o

oo

o0

O

oo

o

O

C

O

oo

o

C

oo

o0

00

oo

O

O

O

O

000000oo

oooo

0o

o0000000000o

0000000000o0

000000000000

ooo

oo

O

O

00

C

O

oo

oo

0oo

o00000000000

o0o

oooooo

000000000000

000000000000

000000000000

oo

O

O

000000000000

oo

0000000000o0

0000000000o0

Q

O

oo

0000000000o0

oo

oo

0000000000oo

ooooooo

ooooooo

oo

0000000000o0

oo

0000000000oo

ooooooo

000000000000

0000000000oo

C

O

oooo

O

000000000000

00000000

oooo

000000000

000000000

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new method for limiting off-target optogenetic stimulation based on Gaussian Process modeling. In holographic photostimulation, the goal is to excite specific neurons via targeted laser light, but widespread expression of opsins may result in additional neurons not in the desired population (\"off-target\" neurons) also firing action potentials. The proposed method uses approximate GP inference (MAP estimation) in combination with a novel gradient-based optimization method to refine target locations in order to minimize the $L_2$ distance between the desired and evoked patterns of stimulation.\n\nThis is a very nice paper that combines good modeling with a very thoughtful approach to experimental realities. While direct experiments will be needed to fully test its efficacy, it stands to make an important contribution to this particular neuroscience application.\n\nSTRENGTHS:\n- Careful modeling of features of real experiments.\n- Use of a scalable GP approach. \n- Clear exposition.\n\nWEAKNESSES:\n- Much of the inference algorithms seems specific to some of the particular modeling choices made (particularly the need for a convex loss in Line 1 of Algorithm 1).\n- Figs 4-5 should clearly be labeled as _simulation experiments_ based on real data; this is clearly explained in ll. 212-214 but elided in, e.g., ll. 254-56, which makes it seem as if the optimization was performed and validated as part of data collection.\n- I was suprised not to see references to Fletcher and Rangan (2014) and Draelos and Pearson (2020) in the related work section.\n\nQUESTIONS:\n- Line 102 defines $\\sigma$ as the sigmoid function. Does that mean logistic? (E.g, $\\tanh$ is also sigmoid in shape but clearly not intended.)\n- It is not stated in the paragraph surrounding Eq. 2 that $I$ is the laser power. It might help to simply state $\\mathbf{x} = (c_1, c_2, I)$.\n- Line 105 states that one should have $g_n(x) \\ge 0$ so that stimulation should always have a positive effect, but in the supplement, this condition is waived in the single-target case. I assume this is because one can compensate in this case by taking $\\theta = \\inf_{x} g_n(x)$? If so, it might help to articulate this in the supplement.\n- what is the MAP version of the ORF $\\hat{g}_n$? This is just $\\hat{g}_n(\\mathbf{X})$, with $\\mathbf{X}$ the matrix of test data points?\n- Why use MSE and not cross-entropy, which would be the standard assumption for a desired set of observations under a Bernoulli model?\n\nLIMITATIONS:\n- Much of the inference algorithm is quite closely tied to the experimental setup. This is both a strength and a limitation, since the proposed work is somewhat unlikely to find broader applications.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe problem the authors tackle in their manuscript is the problem of target selection in holographic optogenetics. Briefly, of the many neurons in a field of view, many experiments require the selective stimulation of a small subset of cells, while minimizing off-target stimulation that may muddy the interpretation of the data. This question is of considerable importance to the field of systems neuroscience, and several experimental methods have been pursued trying to solve it, such as the use of soma-localized opsins or sparse expression of optogenetic protein. The authors present in this manuscript a complementary, Bayesian computational approach that takes into account differences in sensitivity of stimulation of different neurons in a given FOV to optimize laser intensity of position to minimize off-target stimulation while maintaining sufficient target stimulation. The authors use gaussian processes to GPs to infer the response properties of neurons based on stimulation parameters. The authors validate this approach in simulation and in real optogenetic experiments.\n\nSTRENGTHS:\nOriginality: The authors introduce a novel computation algorithm to address a pressing issue in systems neuroscience; that of off-target stimulation. While the authors note that similar methods have been used to infer receptive fields for other types of stimulation, the authors cleverly employ these methods in a new way for inferring the responses of neurons to optogenetic stimulation to address a technical challenge facing the field. This software approach complements experimental approaches and can be readily combined with them. Similar computational approaches of thus far remained lacking, so the work is pioneering in this regard.\n\nClarity: The manuscript is clear with respect to how it poses the question being addressed, how it delineates its approach, and its statement of results.\n\nSignificance: The authors make a significant contribution to a timely question for systems neuroscience and will likely see significance use by labs performing holographic optogenetic stimulation, especially for applications in which minimizing off-target stimulation is crucial. The method could also see use in designing stimulation protocols to precisely pattern the activity of an ensemble of neurons, although that is a future direction.\n\nWEAKNESSES:\nOptimizing stimulation according to one parameter (in this case, minimization of off-target stimulation) often necessitates trade-off to other parameters. The authors bring up one such parameter, the time required to map the ORF and perform the optimization. While this consideration will vary from experimenter to experimenter, as the authors suggest, other considerations might not. For example, by moving stimulation locations off-center relative to the soma, stimulation reliability might become more sensitive to sample motion. Furthermore, the authors assume a ORF fixed during time; the manuscript would benefit from additional consideration for how the ORF changes with sustained stimulation, as desensitization of the channelrhodopsin could significant affect the ORF. In general, the authors should further explore the limitations and trade-offs of this method. This weakness is minor, however, since I assume that these other considerations could be incorporated as parameters to be optimized.\n\nQUESTIONS:\nAre the real-world validations provided using soma-localized channelrhodopsin, or is there channelrhodopsin in the processes? Does the method provide a greater benefit in one situation versus the other?\n\nAs stated in weaknesses section:\nHow sensitive is an OTS-minimized stimulation protocol to sample motion? How does this compare to soma-targeting? How about to changes in neuron sensitivity to stimulation? Can these be incorporated into optimization?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a method for reducing off-target stimulation of neurons during photostimulation by modifying the laser power and target locations. They use a Bayesian optimization approach to determine neuron responses (ORFs) to stimulations at different targets and laser powers and then choose the optimal target parameters to attain a desired neural response to stimulation. In contrast to typical stimulation targeted to the desired neuron’s soma, stimulating somewhat ‘off-target’ can still produce spikes in the desired neurons and avoid stimulating its nearby neighbors.\n\nSTRENGTHS:\nOriginality: This work is well-motivated by the growing number of methods used for direct neuronal stimulation using photostimulation to determine e.g. causal effects of single neuron activity or the functional structure of a neural circuit. Here, rather than address the inference of neural function or connectivity itself, the authors present a solution to the problem of off-target stimulation (OTS), an experimental reality where neural targets not selected for stimulation are stimulated by the laser anyway (due to e.g. poor spatial resolution). The idea of using Bayesian optimization (BO) and finding locations for off-soma stimulations that reduce OTS is very appealing and the method proposed here with BO, GPs to model neural responses, and optimization of stimuli location shows a nice original combination of techniques to address this problem.\n\nQuality: The methods used (GPs, BO, gradient descent for optimization) are all well-established and robust. The inference method for the ORFs with the additional non-negativity constraint is nicely explained. The results clearly demonstrate an advantage of this method over traditional nuclear stimulation for the single-neural-target case in particular. \n\nClarity: Overall the paper is very well written. Ideas from the literature are well-sourced and the authors make it clear how their approach uses some similar techniques and where their method is novel. The figures are good quality as well. The code is relatively clear, though it could use additional documentation or commenting for ease of use. \n\nSignificance: As more neuroscience researchers turn to photostimulation for causal testing of hypotheses, this method should prove useful for the cases where laser accuracy is insufficient for selective stimulations.\n\nWEAKNESSES:\nOne weakness is the lack of experimental data, though it is understandably still difficult to obtain. While the authors mentioned slice data, this was only used as input to their simulations. Looking at the various ORFs mentioned in the supplement, is it reasonable to think that only 4 types of ORFs in the simulation is enough? And if these ORFs that were experimentally obtained were done so via stimulating a grid nearby, would that change the estimation of 1 neuron's ORF if another neuron nearby was also being stimulated (in an ensemble)? In vivo, one would expect that confounding neural activity could occur due to connectivity in addition to laser power spillover. \n\nIt is unclear if, at the end, this method is useful for ensemble stimulation in addition to single target paradigms. While the initial motivation and explanation of the method appears to be geared towards both cases (including Algorithm 1), it is not clear if in Figures 4-5 the optimization of each 'single-target hologram' is the end result of an optimization across multiple targets, or if the optimization was made per target. The different factors examined here ('density', size of ensemble, xy plus z dimensions) are all nicely presented but showing the method works in the 'worst case' (high density, larger ensembles, fully 3D, ...) would make this work stronger.\n\nQUESTIONS:\n1. How well does this scale from an optimization (time) perspective? Can this be done for thousands of neurons? Ensembles of 50?\n\n2. How does the exact neuron expression matter (to the ORF shape)? Can a simplified version be used during the 2nd step of target optimization?\n\n3. Similarly, what do ORFs look like? Are there other references used for establishing the prior beyond the few shown here?\n\n4. Would it be possible to integrate spatial information (from e.g. calcium imaging) into the ORF prior in a useful way? To minimize the number of required mapping stimulations before the target optimization step. \n\n5. It appears that the method uses random locations to first map the ORFs. Would it be possible to optimize the selection of locations for the ORF mapping based on the desired targets for later stimulation?\n\n6. If Algorithm 1 needs to be run many times with different seeds, how long does it take to determine the optimal stimulation pattern? How practical is this in an experimental setup?\n\n7. Clarification on Figure 3: Line 199 says increasing density of neurons, and Fig. 3a, shows 50 - 150 neurons (in the same 250 x 250 um plane) – would be nice to explicitly state that is what was done in simulation, and state what the density therefore is. Why were these density ranges chosen (for different example brain regions)? Are the number of neuron-neuron connections a potential factor here? \n\n8. Is the performance increase the same for lower laser power in the nuclear condition? Ie if we used nuclear stimulation at lower power would we do just as well as optimized? Figure 4 & 5 indicate the optimization chose overall lower laser powers than initial settings.\n\nLIMITATIONS:\nThe authors discussed some trade-offs between the time spent mapping the ORFs and the need for single-cell precision stimulations. They mention potential future work to minimize the time needed for Bayesian target optimization, but fail to detail exact time requirements for the current method.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The authors present a set of methods to efficiently characterize the activation field under optogenetics and optimize the optogenetic stimulation patterns to target certain cells while avoiding the others.

STRENGTHS:
The paper tackles an important problem for reproducing the neural responses with high resolution in a neural circuit. The presented approach is direct, principled and effective.

WEAKNESSES:
The experimental validation of the method is limited. It is unclear how the real-data was used in analysis (Sec 4.2), and if it represents what would happen in a real experiment.

QUESTIONS:
• How is linear summation of activation from different optogenetic stimulation sites justified, especially if the sites are very close to each other?
• Could a simple stimulation optimization approach, where the location is at the intersection of high g(x) for one cell and low g(x) for other cells, work?

A characterization of the efficiency (reduction in the number of measurements for the same estimation quality) is missing.

LIMITATIONS:
Limitations have been adequately addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07