Patent Description:
The result of a regression analysis of sensor data may be applied for various control tasks. For example, in an autonomous driving scenario, a vehicle may perform regression analysis of sensor data indicating a curvature of the road to derive a maximum speed.

However, in many applications, it is not only relevant what the result is (e.g. maximum speed in the above example) but also how certain the result is. For example, in an autonomous driving scenario, a vehicle controller should take into account whether the prediction of a maximum possible maximum speed has sufficient certainty before controlling the vehicle accordingly.

The publication by <NPL> introduces a neural network that governs the dynamics of an Ordinary Differential Equation (ODE) as a generic building block in learning systems. The input pattern is set as an initial value for this ODE. However, this is a fully deterministic dynamical system, hence it cannot express uncertainties.

The publication by <NPL>, introduces nonparametric approach for estimating drift and diffusion functions in systems of stochastic differential equations from observations of the state vector. Gaussian processes are used for these functions and estimates are calculated directly from dense data sets using Gaussian process regression.

The publication by <NPL>, introduces practical methods to capture uncertainties in a 3D vehicle detector for Lidar point clouds.

The publication by <NPL>, introduces a stochastic modeling and planning approach using deep Bayesian neural networks (DBNNs).

The publication by <NPL>), introduces the use of Bayesian networks, which provide both a mean value and an uncertainty estimate as output, to enhance the safety of learned control policies under circumstances in which a test-time input differs significantly from the training set.

The publication by <NPL>, proposes combining adaptive preconditioners with Stochastic Gradient Langevin Dynamics (SGLD).

In view of the above, flexible machine learning approaches which provide uncertainty information for an output are desirable.

The method and the device with the features of the independent claims <NUM> and <NUM> allow achieving improved robustness compared to a deterministic approach by modelling the flow dynamics as a stochastic differential equation (SDE) and quantifying prediction uncertainty. Specifically, robustness is improved by assigning Bayesian neural networks (BNNs) on the drift and diffusion terms of the SDE. By using the BNNs in this manner a second source of stochasticity (in addition to the Wiener process for the diffusion) coming from the BNN weights is introduced which improves robustness and the quality of prediction uncertainty assignments.

Additionally, compared to approaches based on dropout, the method and device according to the independent claims do not require manual dropout rate tuning and provides a richer solution family than fixed-rate dropout.

The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention, as defined by the appended claims.

In the following, various examples will be described in more detail.

<FIG> shows an example for regression in an autonomous driving scenario.

In the example of <FIG>, a vehicle <NUM>, for example a car, van or motorcycle is provided with a vehicle controller <NUM>.

The vehicle controller <NUM> includes data processing components, e.g. a processor (e.g. a CPU (central processing unit)) <NUM> and a memory <NUM> for storing control software according to which the vehicle controller <NUM> operates and data on which the processor <NUM> operates.

In this example, the stored control software comprises instructions that, when executed by the processor <NUM>, make the processor implement a regression algorithm <NUM>.

The data stored in memory <NUM> can include input sensor data from one or more sensors <NUM>. For example, the one or more sensors <NUM> may include a sensor measuring the speed of the vehicle <NUM> and a sensor data representing the curvature of the road (which may for example be derived from image sensor data processed by object detection for determining the direction of the road), condition of the road, etc. Thus, the sensor data may for example be multi-dimensional (curvature, road condition,. The regression result may for example be one-dimensional.

The vehicle controller <NUM> processes the sensor data and determines a regression result, e.g. a maximum speed, and may control the vehicle using the regression result. For example, it may actuate a break <NUM> if the regression result indicates a maximum speed that is higher than a measured current speed of the vehicle <NUM>.

The regression algorithm <NUM> may include a machine learning model <NUM>. The machine learning model <NUM> may be trained using training data to make predictions (such as a maximum speed). Due to the safety issues related to the control task, a machine learning model <NUM> may be selected which not only outputs a regression result but also an indication of its certainty of the regression result. The controller <NUM> may take this certainty into account when controlling the vehicle <NUM>, for example brake even if it is below the predicted maximum speed in case the certainty of the prediction is low (e.g. below a predetermined threshold).

A widely used machine learning model is a deep neural network. A deep neural network is trained to implement a function that non-linearly transforms input data (in other words an input pattern) to output data (an output pattern). If the neural network is as residual neural network, its processing pipeline can be viewed as an ODE (ordinary differential equation) system discretized across even time intervals. Rephrasing this model in terms of a continuous-time ODE is referred to as a Neural ODE.

According to various embodiments, a generic Bayesian neural model is provided (which may for example be used as machine learning model <NUM>) that includes solving a SDE (statistical differential equation) as an intermediate step to model the flow of activation maps. The drift function and the diffusion function of the SDE are implemented as Bayesian neural nets (BNN).

According to a Neural-ODE approach the processing of a neural network is formulated as: <MAT> where <NUM> reflects the parameters of the neural network and ht+<NUM> is the output of layer t+<NUM>. This can be interpreted as the explicit Euler-scheme for solving ODEs with step size <NUM>.

With this interpretation, the above equation can be reformulated as: <MAT>.

Thus, ODE calculus may be used for propagating through the neural network. For making this equation stochastic, stochastic ordinary differential equations are considered. In general form they are given as: <MAT>.

The equation is governed by the drift µ(x(t)), which models the deterministic part, and the diffusion σ(x(t)), which models the stochastic part. For a(Xt,t)=<NUM> a standard ODE is obtained. Solving the above equation requires integrating over the Brownian motion dBt, which reflects the stochastic part of the differential equation. One common and easy approximation method of this differential equation is the Euler-Maruyama scheme: Xt+<NUM> =Xt+µ(Xt)Δt+a(Xt)ΔW.

This approximation also holds when the variable xi is a vector <MAT>. In that case the diffusion term is a matrix-valued function of the input and time <MAT> and corresponding ΔW is modelled as P independent Wiener processes ΔW ~ <IMG>(<NUM>, ΔtIP)with IP as the P-dimensional identity matrix.

As stated above, according to various embodiments, µ(xi, ti) and a(xi, ti) are each provided by a respective Bayesian Neural Network (BNN), wherein the weights of the BNN calculating µ(xi, ti) are denoted by θ<NUM> and the weights of the BNN calculating a(xi, ti) are denoted by θ<NUM>. The weights may be at least partially shared between the
BNNs, i.e. θ<NUM> ∩ θ<NUM> ≠ <NUM≯.

The resulting probabilistic machine learning model can be described by <MAT> <MAT> <MAT>.

The first line is a prior on the SDE parameters (weights of the BNNs in this case), the second line is the solution of an SDE, and the last line is a likelihood suitable to the output space of the machine learning model. T is the duration of the flow corresponding to the model capacity.

<FIG> shows an illustration of the machine learning model.

For the (input observation) vector x as initial condition, a realization of a stochastic process <NUM> representing the continuous time activation maps h(t) is determined as solution of an SDE. The h(t) for all t from <NUM> to T (with e.g. h(<NUM>) = x) can be seen as latent representations of the input pattern x at every time instant t. The part of machine learning model doing this determination is referred to as Differential Bayesian Neural Net (DBNN). It includes BNNs <NUM>, <NUM> providing the mean term and the diffusion term, respectively, of the SDE (each taking h(t) and t as input). The DBNN outputs an output value h(T) (which may be a vector of same dimension as the input vector x).

Depending on the application an additional (e.g. linear) layer <NUM> calculates the output y of the model, e.g. a regression result for the input sensor data vector x. This additional layer <NUM> may particular reduce the dimension of h(T) (which can be seen as end state) to a desired output dimension, e.g. generate a real number y from the vector h(T).

The probability distribution of the stochastic process is given by <MAT>.

However, it is possible to take approximate samples from it by a discretization rule such as Euler-Maruyama.

According to one embodiment, as a work-around, the stochastic process is marginalized out of the likelihood by Monte Carlo integration according to <MAT> where <MAT> is the realization at time T of the mth Euler-Maruyama draw. Having integrated out the stochastic process, the model may be trained by approximate posterior inference problem on p(<NUM><NUM>, <NUM><NUM>|x,y). The sample-driven solution to the stochastic process h integrates naturally into a Markov Chain Monte Carlo (MCMC) scheme. According to one embodiment, Stochastic Gradient Langevin Dynamics (SGLD) with a block decay structure is used to benefit from the gradient-descent algorithm as a subroutine (which is essential to train neural networks effectively.

In the following a training algorithm for the model, i.e. an algorithm for supervised learning to determine θ<NUM> and θ<NUM> from training data (comprising a plurality of minibatches), is described.

It should be noted that the gradient <MAT> may be determined using back propagation. It should further be noted that a probability distribution of θ<NUM> and a probability distribution of θ<NUM> may be determined by storing the values of the latest iterations (e.g. for the last <NUM> i) to arrive at trained BNNs <NUM>, <NUM>.

For regression, an additional linear layer <NUM> is placed above h(T) in order to match the output dimensionality. Since the properties of the distribution p(h(T)|x) can be estimated in terms of a mean m(θ<NUM>) and (a Cholesky decompose of) a covariance L(θ<NUM>) L(θ<NUM>)T = Σ(θ<NUM>). Both moments can be determined and then propagated through the linear
layer <NUM>. The predictive mean is thus modelled as <MAT> and the predictive variance as <MAT>. It is possible to design Lθ<NUM> as a diagonal matrix assuming uncorrelated activation map dimensions.

Further, Lθ<NUM> can be parameterized by assigning the DBNN output on its Cholesky decomposition or can take any other structure of the form <MAT>. When choosing P < D, it is possible to heavily reduce the number of learnable parameters for high dimensional inputs.

In summary, according to various embodiments, a method is provided as illustrated in <FIG>.

<FIG> shows a flow diagram <NUM> illustrating a method for processing sensor data according to an embodiment.

In <NUM>, input sensor data is received.

In <NUM>, starting from the input sensor data as initial state, a plurality of end states, is determined.

This includes determining, for each end state, a sequence of states, wherein determining the sequence of states comprises, for each state of the sequence beginning with the initial state until the end state,.

In <NUM>, an end state probability distribution is determined from the determined plurality of end states.

In <NUM>, a processing result of the input sensor data is determined from the end state probability distribution.

According to various embodiments, in other words, BNNs are used to provide the drift term and diffusion term at each step of solving a stochastic differential equation. The uncertainty information provided by the BNNs (by sampling the BNN weights) in addition to the uncertainty information provided by solving the stochastic differential equation (by sampling the Brownian motion) provides information for the processing result, which is for example a regression result, e.g. for controlling a device depending on the sensor data.

Sensor data may comprise image data from an imaging sensor like e.g. a camera or other types of sensors which may produce image-like data such as a LIDAR (light detection and ranging) sensor, an ultrasonic sensor or a radar sensor. The sensor data may also comprise other types of sensor data like sensor data of a kinematic sensor (e.g. an acceleration sensor).

Each state may be seen as a latent representation of the input pattern, i.e. the input sensor data which gives the initial state, at a respective time instant t. Illustratively, the state at time instant t can be seen as the initial state (of time instant zero) after being processed by a process represented by the stochastic differential equation for a time period of length t. From a neural network perspective, each state may be seen as an activation map.

The approach of <FIG> can be used as a generic building block in all learning systems that map an input pattern to an output pattern. It can serve as an intermediate processing step that provides a rich mapping family, the parameters of which can then be tuned to a particular data set. Wherever a feed-forward neural network can be used, the approach of <FIG> can be used. Further, it is especially useful in safety-critical applications where the predictions of a computer system need to be justified or their uncertainty need to be considered before taking downstream actions depending on this prediction.

In particular, the approach of <FIG> may be applied in all supervised learning setups where a likelihood distribution can be expressed for outputs (e.g. normal distribution for continuous outputs, multinomial distribution for discrete outputs). Further, it may be applied in any generative method where the latent representation has the same dimensionality as the observation. It may further be applied in hypernets that use the resultant BNN weight distribution as an approximate distribution in an inference problem, such as variational inference. Examples for applications are image segmentation and reinforcement learning.

The method of <FIG> may be performed by one or more computers including one or more data processing units. The term "data processing unit" can be understood as any type of entity that allows the processing of data or signals. For example, the data or signals may be treated according to at least one (i.e., one or more than one) specific function performed by the data processing unit. A data processing unit may include an analogue circuit, a digital circuit, a composite signal circuit, a logic circuit, a microprocessor, a micro controller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable gate array (FPGA) integrated circuit or any combination thereof or be formed from it. Any other way of implementing the respective functions, which will be described in more detail below, may also be understood as data processing unit or logic circuitry. It will be understood that one or more of the method steps described in detail herein may be executed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

The first Bayesian neural network and the second Bayesian neural network may be trained by comparing, for each of a plurality of training data units, the processing result for input sensor training data of the training data unit with a reference values of the training data unit.

Generally, the approach of <FIG> may be used to generate control data (e.g. one or more control values) from input sensor data, e.g. data for controlling a robot. The term "robot" can be understood to refer to any physical system or technical system (with a mechanical part whose movement is controlled), such as a computer-controlled machine, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant or an access control system.

Claim 1:
A computer-implemented method (<NUM>) for processing
sensor data, the method comprising: receiving (<NUM>) input sensor data;
determining (<NUM>), starting from the input sensor data as initial state, a plurality of end states, comprising determining, for each end state, a sequence of states, wherein determining the sequence of states comprises, for each state of the sequence beginning with the initial state until the end state,
a first Bayesian neural network (<NUM>) determining a sample of a drift term in response to inputting the respective state;
a second Bayesian neural network (<NUM>) determining a sample of a diffusion term in response to inputting the respective state; and determining a subsequent state by sampling a stochastic differential equation comprising the sample of the drift term as drift term and the sample of the diffusion term as diffusion term;
determining (<NUM>) an end state probability distribution from the determined plurality of end states; and
determining (<NUM>) a processing result of the input sensor data from the end state probability distribution and controlling a movement of a mechanical part of a technical system according to the processing result.