Patent ID: 12214201

DETAILED DESCRIPTION

Described here are systems and methods for adaptively controlling a medical device based in part on a Bayesian optimization of the control parameters of the device. The Bayesian optimization provides automatic tuning of the control parameters of the medical device based on feedback data, such as user response data, to achieve a user-specific therapy or effect. Examples of medical devices that can be adaptively controlled in the manner include electrical stimulation devices, such as peripheral nerve stimulators, central nervous system stimulators, cardiac pacemakers, and cardiac resynchronization therapy (“CRT”) devices; drug pumps; hearing aids; cochlear implants; and other such devices with controllable parameters. Examples of feedback data that can be input to the Bayesian optimization include physiological data, such as neural signals, cardiac signals, and chemical signals (e.g., insulin levels, glucose levels). The feedback data may also include behavioral metrics and user preferences (e.g., preferences based on a questionnaire or between two presented options). To implement user preferences, the user preferences can be made into a response surface using a probit function or other suitable statistical or mathematical model.

As one non-limiting example, an adaptive control of the medical device can be provided. Feedback data can be measured or otherwise provided to a controller, and a Bayesian optimization is implemented with a hardware processor and memory to compute updated control parameters for the medical device based in part on the feedback data.

An adaptive controller for use in medical devices is described in the present disclosure. The adaptive controller is implemented using a hardware processor and memory and generally includes a feedback driven control parameter optimization, as shown inFIG.1.

The feedback driven control parameter optimization of the adaptive controller employs Bayesian optimization to intelligently sample the parameter space and select the optimal set of parameters. In some implementations, the optimization generally operates on a timescale that is on the order of a few seconds, but in some other implementations may operate on longer timescales (e.g., days to weeks). After selecting a new control parameter set, the optimization can wait a delay time in order to allow the subject to settle into a steady state. As an example, the delay time can be on the order of 10 seconds. In some embodiments, the optimization can then estimate one or more measurement parameters from the subject. As one non-limiting example, in DBS applications the optimization can estimate the amplitude of the beta oscillations, or other neuronal signals, over a measurement time. For instance, the amplitude can be estimated by keeping a running average of the oscillations amplitude. The measurement time can be on the order of 10 seconds as well.

Bayesian optimization is well-suited for selecting the optimal parameters of a controllable medical device, such as an electrical stimulation device, drug pump, or hearing aid. For instance, Bayesian optimization offers advantages because direct access to the objective function is often not available for such devices and, thus, noisy observations are made instead; the objective function is expensive to evaluate; there is no access to derivatives; and the optimization problem is not necessarily convex.

In the case of electrical stimulation, the objective function is the user's response to a set of feedback stimulator parameters. It may take seconds, minutes, hours, or even days to obtain a good (likely noisy) measure of the effect of a parameter set. Additionally, in DBS and other electrical stimulation applications there is typically no access to derivatives, and so gradient descent methods cannot be used, and the problem at hand cannot be assumed to be convex, so a global exploration is preferred.

Bayesian optimization address each of these challenges. The Bayesian optimization algorithm generally includes treating the unknown objective function as a random function over which a prior is placed. The prior generally represents the believed behavior of the unknown objective function. This prior is updated based in part on feedback data that are representative of evaluations of the unknown objective function to form a posterior distribution over the unknown objective function. The posterior distribution is then used to generate an acquisition function, which models the utility of sampling across the space. The acquisition function is used to determine the next sample points. As one non-limiting example, the posterior distribution can be a Gaussian process regressor (“GPR”) that estimates the objective function with a response surface, and the acquisition function predicts the utility of sampling by incorporating the mean and variance of the GPR.

A Gaussian process is an extension of the multivariate Gaussian distribution to an infinite-dimension stochastic process. A Gaussian process can be thought of as a distribution over functions, specified by a mean function, m, and covariance function (also known as the kernel), k:
f(x)˜GP(m(x),k(x,x′))  (1).

The prior mean is often assumed to be the zero function m(x)=0, or the mean of the training data, m(x)=f. One example choice is the squared exponential function,

k⁡(xi,xj)=σ2⁢exp⁡(-xi-xj22⁢ℓ);(2)

where σ2is the variance andis a length scale parameter. In the Bayesian optimization task, the GP is fit to previous observations, D1:n={xi:n, fi:n}, in order to obtain the posterior for any point xn+1. The predictive distribution can be derived as,
P(fn+1|D1:n,xn+1)=N(μn(xn+1),σ2(xn+1))  (3);
where
μn(xn+1)=kTK−1f1:n(4);
σn2(xn+1)=k(xt+1,xt+1)−kTK−1k(5).

Thus, given a set of previous observations, D, the mean and variance of any point xn+1can be predicted, which can be used to determine which point should be sampled next using the acquisition function.

The acquisition function, u( . . . ), serves to guide the search to the optimum by modeling the expected utility of sampling at any point, xn+1. An acquisition function such as the Gaussian process lower confidence bound (“GP-LCB”) function achieves low values in regions where either the prediction is low, the uncertainty is high, or both.
GP−LCB(x)=μ(x)−κσ(x)  (6);

where κ≥0. Other acquisition functions can also be used. The Bayesian optimization algorithm thus selects the next evaluation point, xn+1, by minimizing the acquisition function, such as by sampling at argmaxxu(x|D).

The acquisition function also governs the trade-off between exploration and exploitation. In GP-LCB, the κ parameter determines the exploration-exploitation trade-off, where high values of κ encourage exploration and low values of κ encourage exploitation.

With,

κt=v⁢⁢τt,⁢v=1,andτt=2⁢log⁡(td2+2⁢π23⁢δ),

it can be shown that this method is no regret with high probability.FIG.2shows an example run of a Bayesian optimization algorithm on a 1D problem. The optimization starts with 3 points, from which it fits a Gaussian process regressor. The Bayesian optimization algorithm then computes an acquisition function from the GPR, which incorporates both the mean and variance of the GPR, to model the utility of sampling. The Bayesian optimization then minimizes the acquisition function to determine where to sample next. Finally, the objective function is sampled, and the process is repeated. It will be appreciated by those skilled in the art that other forms of Bayesian optimization (e.g., different Bayesian models, different acquisition functions) other than those described here can also be implemented with the systems and methods described in the present disclosure.

In general, feedback data provided to the adaptive controller can be used to generate the response surface estimated by the GPR. As noted above, the feedback data can include physiological feedback data measured from the user, which may include electrophysiological signals, chemical signals, and so on. Such physiological feedback data can be measured by the controllable medical device, or can be provided to the controllable medical device via an input. For instance, the physiological feedback data could be measured by a different medical device and transmitted over a wired or wireless connection to the adaptive controller used for the controllable medical device.

As mentioned above, in some embodiments, the feedback data can include behavioral or user preference data that can be quantified and provided as an additional input to the Bayesian optimization. For instance, the user can qualitatively describe their interpretation of whether a current batch of settings for the controllable medical device are effective or not. As one example, the user preference data can be recorded as responses to a questionnaire. This qualitative feedback can be used to build out a Gaussian process to be used in the Bayesian optimization. As one example, a probit function may be used to build the GPR. A probit function may be, for instance, a regression-based model where the dependent variable can take only two values. Within the context of the systems and methods described in the present disclosure, the probit function may be built using dependent variables that are associated with user preferences on control parameter settings. For instance, the user preferences could be a user's preferred choice between two different control parameter settings. The preference could be a forced choice (e.g., better/worse), or better/same/worse, or much better/better/worse/much worse, or so on.

The user's specific trade-off between exploration and exploitation can also be incorporated into the Bayesian optimization. For instance, some users may be more willing to try more therapy and thus that user's tolerance for exploitation versus exploration may be different from other users.

Implementing user preference to design the control parameter settings for the controllable medical device effectively amounts to a changing cost function over time. The methods described in the present disclosure are capable of optimizing this changing cost function to select the settings that are contained within the current set of user preferences.

In an example study, the adaptive controller described in the present disclosure was tested for controlling a neurostimulator used in a DBS application. In this example, an adaptive dual controller (“ADC”) was implemented using a hardware processor and memory. The ADC generally included an inner loop and an outer loop, as shown in the example configurations inFIGS.3A and3B. The inner loop was a parameterized feedback control loop, and the outer loop was a parameter adjustment loop that implemented the Bayesian optimization described in the present disclosure.

In this example, the inner loop of the ADC included a closed-loop feedback stimulator. The inner loop received information from a model that was based on a local field potential (“LFP”) of the GPi, as shown inFIG.4. The model shown inFIG.4is a physiologically realistic mean-field model (“MFM”) of the basalganglia thalamocortical system (“BGTCs”). The BGTCs MFM models the mean firing rate and voltage of nine cortical and subcortical structures using two second-order differential equations.

DBS was added to the model by representing a DBS pulse as a direct current injection into the target structure. Integrating Ohm's law for capacitors results in the following:

Δ⁢V=Δ⁢⁢t·iC.(7)

DBS was therefore modeled by directly adding the total charge of a monophasic DBS pulse (divided by the membrane capacitance) to the first derivative of the voltage.FIG.5shows example voltage traces from the GPi in the nave, PD, and PD with DBS states, as well as power spectrum from each trace.

Referring again toFIG.3A, the inner loop extracts phase and amplitude information from the LFP in real time. In some embodiments, then, the inner loop has three parameters: an oscillation phase threshold, an oscillation amplitude threshold, and a stimulus amplitude. In other embodiments, different parameters can be extracted and thus the inner loop can have different parameters. In general, the inner loop operated on a short timescale, such as 1 ms.

As shown in the example inFIG.3B, the adaptive dual controller can be configured to have dual goals (exploitation and exploration), and can be composed of two loops: an inner parameterized stimulator and an outer parameter adjustment loop. The inner loop may incorporate feedback from the user to alter stimulation. The outer loop is composed of an estimator and a design block, and is given a specification. The estimator builds a model of the relationship between stimulation parameters and some measure of user outcome, which it passes on to the design block. The design block then incorporates this information with the specification to select new parameters for the inner loop. The inner loop operates on a much shorter timescale than the outer loop.

To implement phase and amplitude feedback stimulation, a real-time method of accurately estimating both phase and amplitude of an oscillation can be used. As one example, phasic stimulation has been accomplished by band-pass filtering the signal and then using the time since the preceding zero crossing. Amplitude-based stimulation has been achieved by rectifying and smoothing the band-passed signal for 400 ms. A visualization of amplitude feedback stimulation, phase feedback stimulation, and combined phased/amplitude feedback stimulation is shown inFIG.6.

As another example, a sliding Fourier transform technique referred to as the Sliding Windowed Infinite Fourier Transform (“SWIFT”) and described in co-pending U.S. Provisional Patent Appln. No. 62/520,265, which is herein incorporated by reference in its entirety, can be used. Unlike other methods of phase/amplitude estimation, the SWIFT technique directly and efficiently calculates the windowed Fourier transform of the signal, centered on ω=2πf/fsand windowed with an infinite length, causal exponential window. In a variation referred to as αSWIFT, the α function window, which is the difference between two exponentials with different time constants, is used to achieve improved frequency resolution.

The SWIFT algorithm has two parameters that control its behavior: the center frequency, ω; and the time constant, τ. For the αSWIFT algorithm two time constants are used: τslowand τfast. The center frequency, ω, can be set to match frequency information of the model. For instance, the center frequency can be set to match the center frequency of a beta peak in the model. The τ time constant (τslowin αSWIFT) controls the time-frequency tradeoff: a shorter time constant leads to higher temporal resolution, but lower frequency resolution (i.e., wider frequency response). As one example of tuning this tradeoff, the width of the SWIFT algorithm's frequency response can be matched to the width of the model's beta peak at −6 dB (or 50% power reduction).FIG.7shows an example of the phase amplitude feedback stimulation algorithm operating on an example LFP, in which stimulation is triggered off both phase and amplitude.FIG.8shows a comparison of the Bayesian optimization described in the present disclosure working on the model used in this example, and shows how the Bayesian optimization (blue lines) performs relative to brute force optimization and Nelder-Mead (orange lines).

FIGS.9A-9Bshow another example of a Bayesian adaptive dual controller.FIG.9Ashows an example of the Bayesian ADC control diagram. In this example, the Bayesian ADC's inner loop was composed of a phase/power based feedback stimulator. The outer Bayesian optimization loop was composed of a Gaussian process (GP), and acquisition function. The Gaussian process builds a model of how the stimulation parameters affect the feedback signal, and the acquisition function uses this information to select the next parameter set.FIG.9Bshows an overview of the Bayesian ADC's cyclic operation. The Bayesian ADC sets the stimulator parameters and applies phase/power based stimulation to the BGTCS for 20 s. It then estimates the effect of those parameters on beta power, and updates its GP with the new observation. Finally, it optimizes its acquisition function, and selects the next parameter set.

Referring now toFIG.10, an example of a controller1010that can implement the methods described in the present disclosure to adaptively control a controllable medical device is illustrated. In general, the controller1010includes a processor1012, a memory1014, and input1016, and an output1018. The controller1010can be implemented as part of a controllable medical device, or as a separate controller that is in communication with the controllable medical device via the output1018. As one example, the controller1010can be implemented in a controllable medical device, such as an implantable medical device (e.g., an implanted nerve stimulation system or an implanted cardiac rhythm management system), a hearing aid, and so on. In other examples, the controller1010can be implemented in a remote computer that communicates with the controllable medical device. In still other example, the controller1010can be implemented in a smartphone that is paired with the controllable medical device, such as via Bluetooth or another wireless or wired communication.

In some embodiments, the input1016is capable of sensing feedback data from the user. As one example, the feedback data can be electrophysiological activity, and the input1016can be one or more electrodes. As another example, the feedback data can be chemical signal data, such as measured levels of chemicals. In such instances, the input1016can include a suitable sensor for measuring the chemical signal data. As noted above, such a sensor could be a part of the controllable medical device, or could be a separate sensor that is in communication with the controller1010via the input1016, whether through a wired or wireless connection. Such chemical data could also be measured through other means, such as via a blood sample taken from the user, and transmitted to the controller1010via the input1016. The input1016can thus more generally include a wired or wireless connector for receiving feedback data, which as noted above may also include behavioral or user preference data. In these latter examples, the feedback data can include a response surface generated from the behavioral or user preference data, such as a probit function, that is transmitted to the controller1010via the input1016.

The processor1012includes at least one hardware processor to execute instructions embedded in or otherwise stored on the memory1014to implement the methods described in the present disclosure. The memory can also store measured feedback data for processing, as well as settings to be provided to the processor1012for generating control signals to be provided to a controllable medical device via the output1018. As described above, these settings can be stored and also updated by the adaptive control implemented by the controller1010.

The output1018communicates control signal to a controllable medical device. As one example, where the controllable medical device is an electrical stimulation device, the control signals provided to the output1018can control one or more electrodes to operate under control of the controller1010to sense electrophysiological activity in a subject and to deliver electrical stimulations to the subject in response thereto. Sensing circuitry in the controller1010can detect and processes electrophysiological activity sensed by the one or more one electrodes via the input1016to determine the optimized stimulation settings (e.g., phasic burst stimulation settings) based on the methods and algorithms described above. The optimized settings are provided as instructions to a pulse generator in the electrical stimulation device via the output1018, which in response to the instructions provides an electrical signal to the one or more electrodes to deliver the electrical stimulations to the subject.

The controller1010can also include a transceiver1020and associated circuitry for communicating with a programmer or other external or internal device. As one example, the transceiver1020can include a telemetry coil. In some embodiments, the transceiver1020can be a part of the input1016.

In operation, the controller1010receives feedback data from the subject via the input1016. These feedback data are provided to the processor1012where they are processed. For example, the processor1012analyzes the feedback data and generates an appropriate response surface, or otherwise generates a GPR to be used for the Bayesian optimization to update the control parameter settings for the controllable medical device.

In one non-limiting example, the processor1012can process electrophysiological signal data to estimate biomarkers such as amplitude data, phase data, or both, from the measured data. In these instances, the processor1012can analyze the electrophysiological signals using a SWIFT or αSWIFT algorithm to extract the relevant amplitude and phase data. In other example, other biomarkers can be extracted or estimated from the electrophysiological signals, such as phase-amplitude coupling, evoked compound action potentials, or other parameters or characteristics that can be extracted or estimated from the electrophysiological signals. The extracted biomarkers are then input to a Bayesian optimization algorithm implemented by a hardware processor and memory1014of the controller1012to determine optimized settings for the delivery of electrical stimulation to the subject, as described above in detail. The optimized settings are provided to the electrical stimulation device via the output1018to control one or more electrodes to generate electrical stimulation that will achieve the desired effect in the subject, such as preventing an anticipated pathological electrophysiological event.

The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.