Learning Density Distribution of Reachable States for Autonomous Systems
Anonymous Author(s)
AfﬁliationAddress email
Abstract: State density distribution, in contrast to worst-case reachability, can be leveraged for safety-related problems to better quantify the likelihood of therisk for potentially hazardous situations.
In this work, we propose a data-drivenmethod to compute the density distribution of reachable states for nonlinear andeven black-box systems.
Our semi-supervised approach learns system dynamicsand the state density jointly from trajectory data, guided by the fact that the statedensity evolution follows the Liouville partial differential equation.
With the helpof neural network reachability tools, our approach can estimate the set of all possible future states as well as their density.
Moreover, we could perform onlinesafety veriﬁcation with probability ranges for unsafe behaviors to occur.
We usean extensive set of experiments to show that our learned solution can produce amuch more accurate estimate on density distribution, and can quantify risks lessconservatively and ﬂexibly comparing with worst-case analysis.
Introduction
[Figure]
Reachability analysis has been a central topicand the key for the veriﬁcation of safety-criticalautonomous systems.
The majority of existingreachability approaches either compute worstcase reachable sets [1, 2, 3, 4, 5, 6], or use
Monte Carlo simulation to estimate the reachable states with probabilistic guarantees [7, 8,9, 2].
There are several obvious disadvantagesof such methods. 1).
Worst-case methods, especially those for nonlinear systems, often overapproximate the reachable sets and produce very conservative results (due to convex set representations [10, 11, 12, 13], wrap effects [14, 15], low-order approximation [3, 16], etc.), not to say theyare usually very computationally expensive (also known as the curse of dimensionality). 2).
Existing methods do not care about reachable state concentration and produce the same reachable stateestimations for different distributions of initial states if those distributions share the same support 1 .
That is, current approaches do not compute which states are more likely to be reached.
For example,in Fig.
1, the state density distribution of a Van der Pol oscillator evolves over time and concentrateson certain states (the highlighted part in the ﬁgure), even if the initial states are uniformly distributed.
If one uses an existing worst-case reachability algorithm, most likely the results in Fig.
1 (b)(c) willshow almost the entire black region inside the highlighted part is reachable, as those methods oftenuse convex sets to represent the reachable sets.
In this paper, we aim to tackle the above problems and propose a learning-based method that cancompute the density distribution of the reachable states from given initial distributions.
The state density is much more powerful than worst-case reachability and can better quantify risks.
Ourproposed method is based on the Liouville theorem [17, 18, 19], which is from classical Hamiltonianmechanics and asserts that the state density distribution function (which is the measurement of stateconcentration) is constant along the trajectories of the system.
Given an autonomous system ˙ x =f ( x ) that is locally Lipschitz continuous, the evolution of the state density ρ ( x, t ) (i.e. the densityof state x at time t ) is characterized by a Liouville partial differential equation (PDE).
We learnthe state density as a neural network (NN) while respecting the laws of physics described by thenonlinear Liouville PDE, in a way that is similar to the physics-informed NN [20].
Furthermore, we make two major improvements to make the learned density NN suitable for veriﬁcation. 1).
Instead of learning ρ for a ﬁxed initial distribution ρ ( x 0 , 0) , we learn the densityconcentration function which speciﬁes the multiplicative change of density from any ρ ( x 0 , 0) .
We use a single NN to jointly learn both the reachable state Φ( x 0 , t ) and its corresponding density.
Moreover, we use Reachable Polyhedral Marching (RPM) [21]—an exact ReLU NN reachabilitytool—to parse our learned NN as linear mappings from input polyhedrons to output polyhedrons.
Using such parsed polyhedrons, we can perform online forward and backward reachability analysisand get the range of density bounds [ ρ min , ρ max ] for each output polyhedron.
Together, our methodcan perform online safety veriﬁcation by computing the probability of safety (instead of a single yesor no answer) under various initial conditions, even with unknown system dynamics f
(i.e. a blackbox system where we only have access to a simulator of x ( t ) ).
In this way, it also has the potentialto collect environmental data in run-time and update its distribution for online safety veriﬁcation.
We conduct experiments on 10 different benchmarks covering systems from low dimension academic examples to high dimension black-box simulators equipped with either hand-crafted or NNcontrollers.
Surprisingly, without using any ground-truth density data in the learning process, ourapproach can achieve up to 99.89% of error reduction in KL divergence with respect to the groundtruth value, when compared to sampling-based methods like kernel density estimation and Gaussian
Processes.
Moreover, our learned density concentration function can also be used for reachabilitydistribution analysis.
We show that in several systems, more than 90% of the states can actuallyjust reside in a small region (less than 10% of the volume of the convex hull for those states) inthe state space, which also points out the conservativeness of the worst-case reachability analysismethods in terms of quantifying risks.
We also show that different initial distributions can lead to adrastic change in the safety probability, which can help in cases when unsafe is inevitable but can bedesigned to happen with a very low probability.
Our major contributions are: (1) we are the ﬁrst to provide an explicit probability density functionfor reachable states of dynamical systems characterized as ordinary differential equations; this density function can be used for online safety analysis, (2) we propose the ﬁrst data-driven method tolearn the state density evolution and give accurate state density estimation for arbitrary (boundedsupport) initial density conditions without retraining or ﬁne-tuning, and (3) we use a variety of examples to show the necessity to perform reachability distribution analysis instead of pure worst-casereachability, to ﬂexibly and less conservatively quantify safety risks.
Related work Reachability analysis, especially worst-case reachability using sampling-basedmethods or set propagation, has been studied for decades.
The literature on reachability has beenextensively studied in many surveys [1, 6, 22, 23].
Here we only discuss a few closely related works.
Hamilton Jacobian PDE has been used to derive the exact reachable sets in [24, 1, 5].
However,the HJ-PDE does not provide the density information.
Many data-driven approaches can compute probabilistic reachable sets using scenario optimization [7, 25], convex shapes [8, 9, 26],support vector machines [27, 28], kernel embedding [29], active learning [30], and Gaussianprocess [2].
However, the probabilistic guarantees they provide are usually in the form that
Prob ( x ( t ) ∈ estimated set ) > 1 − (cid:15) with enough samples, instead of the state density distribution.
[31] estimates human state distribution using a probabilistic model but requires statediscretization.
[32] uses the Liouville equation to maximize the backward reachable set for onlypolynomial system dynamics.
In [33] the authors compute the stochastic reachability by discretizing stochastic hybrid systems to Markov Chains (MC), then perform probabilistic analysis on thediscretized MC.
The closed-form expression of the probability requires integrals over the whole statespace hence is computation-heavy and cannot be used for online safety check.
The closest to oursis [34] where Perron-Frobenius and Koopman operators are learned from samples of trajectories.
Then, the learned operators can be used to transform the moments of the distribution over time.
The distribution at time t is then recovered from the transformed moments.
However, as they use moments up to a speciﬁc order to represent a distribution, even if the learned operators are perfect, theestimation error of the distribution might not be zero.
Also, such a moment-based method is hard toscale to large-dimensional systems.
Recently, there is also a growing interest in studying the (worstcase) reachability of NN [35, 36, 37, 38, 21] or systems with NN controllers [39, 4, 40, 41, 42].
Inthis paper, we use [21] to parse our learned NN as a set of linear mappings between polyhedrons foronline reachability computation, but this step can be replaced with other NN reachability tools.
To measure the probability distribution in the reachable sets, the most na¨ıve approach is to use histograms or kernel density estimation [19], similar to the Monte-Carlo method used in dispersionanalysis [43].
But this could lead to poor accuracy and computational scalability [44].
Another approach propagates the uncertainty by approximating the solution of the probability density function(PDF) transport equation [45], which could still be time-consuming due to the optimization processperformed at each time step.
Our approach ﬁnds the PDF transport equation by solving the Liouville PDE 2 using NN.
Similar ideas have been explored in [18].
However, the density NN in [18]was learned solely for a ﬁxed initial states distribution and therefore, cannot be used for online prediction.
Instead, we jointly learn the reachable states and density changes, and can perform onlinereachability density computation for any initial states distribution.
The idea of using deep learning to solve PDEs can be traced back to the 1990s [46, 47, 48],where the solutions of the PDE on a priori ﬁxed mesh are approximated by NN.
Recently,there is a growing interest in the related sub-ﬁelds including: mesh-free methods in strongform [20, 49, 50] and weak form [51, 52], solving high-dimension PDEs via BSDE method [53],solving parametric PDE [54, 55] and stochastic differential equations [56, 57], learning differentialoperators[58, 59, 60], and developing more advanced toolboxes [61, 62, 63].
Our idea for solving
Liouville PDE along trajectories is similar to [53] but without the stochastic term.
Preliminaries
We denote by R , R ≥ 0 , R n , R n × n the sets of real numbers, non-negative real numbers, n -dimensionalreal vectors, and n × n real matrices.
We consider autonomous dynamical systems of the form˙ x ( t ) =
f ( x ( t )) , where for all t ∈ R , x ( t ) ∈ X ⊆
R d is the state and X is a compact set thatrepresents the state space.
We assume that f : R d (cid:55)→ R d is locally Lipschitz continuous.
Thesolution of the above differential equation exists and is unique for a given initial condition x 0 .
Wedeﬁne the ﬂow map Φ : X × R (cid:55)→ X such that Φ( x 0 , t ) (also written as x ( t ) for brevity) is thestate at time t starting from x 0 at time 0 .
Note that system parameters can be easily incorporated asadditional state variables with time derivative to be 0 .
We analyze the evolution of the dynamical system by equipping it with a density function ρ : X × 128 R
→
R ≥ 0 , which measures how states distribute in the state space at a speciﬁc time instant.
Alarger density ρ ( x, t ) means the state is more likely to reside around x at time t , and vice versa.
The density function is completely determined by the underlying dynamics (i.e., function f ) andthe initial density map ρ 0 : X (cid:55)→ R ≥ 0 .
Speciﬁcally, given a ρ 0 , the density function ρ solves thefollowing boundary value problem of the Liouville PDE [17]:
[Equation]
1) where ∇ · ( ρ · f ) = (cid:80) di =1 ∂ [ ρ · f ] i ∂ [ x ] i is the divergence for the vector ﬁeld ρ · f , and [ · ] i takes thei -th coordinate of a vector.
Intuitively, as shown in [17], Liouville PDE is analogous to the massconservation in ﬂuid mechanics, where the change of density ∂ρ∂t at one point is balanced by thetotal ﬂux traversing the surface of a small volume surrounding that point.
It is hard to solve the
Liouville PDE for a closed from of the density function ρ .
However, it is relatively easy to evaluatethe density along a trajectory of the system.
To do that, we ﬁrst convert the PDE into an ODE asfollows.
Considering a trajectory Φ( x 0 , t ) , the density along this trajectory is an univariate function of t , i.e., ρ ( t ) =
ρ (Φ( x 0 , t ) , t ) .
If we consider the augmented system with states [ x, ρ ] , from Eq.
(1)we can easily get the dynamics of the augmented system [19]:
[Equation]
Therefore, to compute the density at an arbitrary point x T ∈ X at time T , one can simply proceedas follows:
First, ﬁnd the initial state x 0 = Φ( x T , − T ) using the inverse dynamics − f .
Then,solve the Eq.
(2) with initial condition [ x 0 , ρ 0 ( x 0 )] .
The solution at time T just gives the desireddensity value.
However, such a procedure only gives the density at a single point, therefore, cannotbe used in reachability analysis.
Instead, we need to compute the solution of Eq.
(2) for a set ofinitial conditions and use that to compute the reachable sets.
To achieve this, we use an NN with
ReLU (Rectiﬁed Linear Unit) activation functions [64] to jointly approximate the ﬂow map Φ andthe density concentration function , as will be shown in the next section.
Density learning and online reachability density computation
Let us take a closer look at Eq.
For an initial condition [ x 0 , ρ 0 ( x 0 )] , the closed formof the solution ρ is ρ (Φ( x 0 , t ) , t )
=
ρ 0 ( x 0 ) exp (cid:16) − (cid:82) t 0 ∇ · f (Φ( x 0 , τ )) dτ (cid:17) .
Interestingly, thesolution of ρ is linear in the initial condition ρ 0 .
The gained part denoted by G ( x 0 , t ) :=exp (cid:16) − (cid:82) t 0 ∇ · f (Φ( x 0 , τ )) dτ (cid:17) is a function of the initial state x 0 and time t , but is independentof ρ 0 .
This mapping G is completely determined by the underlying dynamics and we call it thedensity concentration function .
With G in hand, the density at an arbitrary time and state can bequickly computed from any ρ 0 .
However, G is obviously hard to compute.
Therefore, we use NN toapproximate the density concentration function .
In addition to G , we also use NN to learn the ﬂowmap Φ , which will be a necessity for computing the reachable set distribution shown in Sec. 3.2.
3.1 System dynamics and density learning framework
Let the parameterized versions of the ﬂow map and the density concentration function be Φ ω and
G θ respectively, where ω and θ are parameters.
To train the neural network, we construct a datasetby randomly sampling N trajectories of the system: D train = { ξ i } N − 1 i =
in T time steps (with timeinterval ∆ t ): ξ i = { ( x i 0 , 0) , ( x i 1 , ∆ t ) , ..., ( x iT − 1 , ( T − 1)∆ t ) } where x ij =
Φ( x i 0 , j ∆ t ) .
Then, thegoal of the learning is to ﬁnd parameters ω
and θ satisfying
[Equation]
where the ﬁrst constraint is for the ﬂow map estimation, and the second constraint enforces the
Liouville equation for all the data points.
As for the implementation, we model Φ ω and G θ jointly as a fully-connected neural network NN ( · , · )with ReLU activations.
To ensure numerical stability as G is an exponential function, we add anonlinear transform from the NN output to the density concentration function :
[Equation]
where NN [ i : j +1] is to choose the i, i + 1 , ..., j -th dimensions from the output of the NN, and z is theintermediate density estimation from the NN.
In this way, we guarantee that the density concentration function at t = 0 is always 1.
We optimize our NN via back propagation with the loss function:
[Equation]
where the ﬁrst term denotes the state estimator square error [65], the second term indicates howfar (in the sense of L2-norm) the solution deviates from the Liouville Equation, and λ balancesthese two loss terms.
We approximate the time derivative of the density concentration function by
˙ G θ ( x i 0 , k ∆ t ) = (cid:2)
G θ ( x i 0 , ( k + 1)∆ t ) − G θ ( x i 0 , k ∆ t ) (cid:3) / ∆ t .
Our method can also work for blackbox systems if we approximate ∇ · f ( · ) numerically.
With some tools from statistical learningtheory, we can show that with a large enough number N of samples, the learned ﬂow map and thedensity concentration function can be arbitrarily accurate.
A formal proof is shown in Appendix A.
3.2 System reachable set distribution computation via NN Reachability Analaysis
In the last section, we have learned an NN that can estimate the density at a single point given theinitial density.
In this section, we will boost this single-point estimation to set-based estimationby analyzing the reachability of the learned NN.
To compute the set of all reachable states and thecorresponding density from a set of initial conditions, we use the Reachable Polyhedron Marching(RPM) method [21] to further process the learned NN for G and Φ .
RPM is a polyhedron-basedapproach for exact reachability analysis for ReLU NNs.
It partitions the input space into polyhedralcells so that in each cell the ReLU activation map does not change and the NN becomes a ﬁxed afﬁnemapping.
With the cells and the corresponding afﬁne mapping on each cell, the exact reachable setfor a set of input can be quickly evaluated.
Backward reachability analysis can also be performed bycomputing the intersection of the pre-image of a query output set with the input polyhedron cells.
System forward reachable set with density.
Recall that the input of the NN in Sec.
3.1 consists ofthe initial state x 0 and time t .
For the simplicity of comparison with other methods, we only estimatethe reachable set and the density at given ﬁxed time instances t and ﬁx the last element of the inputof the NN.
Thus for each time step t, the input polyhedral cells generated from the RPM will bea set of linear inequality constraints on x , and those input cells together with the set of the afﬁnemappings and output polyhedral cells can be represented as: { ( A k , b k , C k , d k , E k , f k ) } Nk =1 , wherein each input cell H
k := { v ∈ R d | A k v ≤ b k } , the NN becomes an afﬁne mapping y
=
C k v + d k ,and thus the image of the input cell is also a polyhedron M k
= { y ∈ R d +1 | E k y ≤ f k } .
Recall that the ﬁrst dimension of our NN output estimates the density concentration functionz , and the rest dimensions estimate the state x , thus the output cell can be written as M k
={ ( z, x ) | E k
[ z, x ] T ≤ f k } .
By projecting it to the state space, we get the reachable set of the system,i.e., R ok :=
{ x ∈ X| ( z, x ) ∈ M k } .
Then, in each cell M k , we evaluate the lower and upper boundsof z , and denote them by z k,min and z k,max .
The density bound for cell M k is then computed as
[Equation]
where ¯ x k is the center of H k .
Finally the system forward reachable set is a union of projected outputpolyhedral cells: N [Equation]
k =1 R ok where each cell R ok is associated with a density bound [ ρ k,min , ρ k,max ] .
System reachable set probability computation.
Given an initial state distribution, we want toﬁgure out the probability distribution of the system forward reachable sets, as well as the probabilityfor the states land into a query set (e.g., the query set could be the unsafe region).
For an arbitrary initial probability density function (whose support is bounded), we can apply RPMto partition its support into cells 4 as in the last section.
Finally, we obtain the reachable sets withbounded state densities { ( H k , ρ mink , ρ maxk ) } Nk =1 and the probability bound in each cell is:
[Equation]
where Vol ( · ) computes the volume for a polyhedron.
By computing for all input cells { H k } Nk =1 ,we can derive the system forward reachable set and the corresponding probability bound as 216 (cid:8) ( H k , P mink , P maxk ) (cid:9) k
=1: N .
The backward reachable set probability can be computed in a similarfashion, by checking the intersection between the query output region and the output cells derived by
RPM, computing each intersection’s probability range by its volume and density bound and ﬁnallyaggregating the probability of all intersections.
Detailed computation is shown in Appendix B.
The RPM method cannot handle systems with higher than 4 dimensional state space in our experiments.
But the technique discussed in Sec.
3.2 can work with any set-based NN reachability tools with slight modiﬁca-
[Figure]
Experimental evaluation in simulation
Here we show the beneﬁts of learning density concentration function from Liouville PDE in statedensity and reachable set distribution estimation.
More experiments are provided in Appendix C ∼ H.
4.1 Implementation details
Datasets: We investigate 10 benchmark dynamical systems as reported in Table 1.
These benchmark systems range from low dimension academic models (2 ∼ 3 dimensions) to complex and evenblack-box systems (13 ∼ 16 dimensions) controlled by handcrafted or NN controllers.
All controllers(except for the ground robot navigation example) are from the original references.
More details (themodel description and initial distributions) will be provided in Appendix C.
Training:
For each system, we generate 10k trajectories through simulation with varied trajectorylengths from 10 to 100 time-steps, depending on different conﬁgurations of the simulation environment.
We use 80% of the samples to train the NN as described in Sec.
3.1 and use the rest forvalidation.
For the trajectory data, we collect the system states and compute for the system divergence term.
For black-box systems, we use the gradient perturbation method to approximate thederivatives.
We use feed-forward NN which has 3 hidden layers with 64 hidden units in each layer.
We use PyTorch [71] to train the NN and the training takes 1 ∼ 2 hours on an RTX2080 Ti GPU.
4.2 Density estimation veriﬁcation
We ﬁrst test the density estimation accuracy of our learned NN.
We compare our approach withother baselines including kernel density estimation (KDE), Sigmoidal Gaussian Process Density(SGPD) [72] and the histogram approach.
For each simulation scenario, we ﬁrst solve Eq.
(2) togenerate 20k ∼ 100k trajectories of (state, density) pairs, and treat this density value as the groundtruth.
For the KDE method, we choose an Epanechnikov kernel.
We then measure the KL divergencebetween the density estimate of each method and the ground truth.
As shown in Fig.
2, our approachhas consistently outperformed KDE, SGPD and histogram approaches, with the largest reduction of99.69% in KL divergence when compared with the histogram approach for the Kraichnan-Orszagsystem, while our method doesn’t use any ODE generated density values during training.
Also inhigh-dimension systems (dimension ≥ 7 ), the histogram approach fails to predict the density due tothe curse of the dimensionality, whereas our approach can always predict the density, with a 30.13%to 99.87% decrease in KL divergence comparing to KDE.
More plots will be given in Appendix D.
4.3 Reachable set distribution analysis
Forward reachable set distribution analysis.
Being conﬁdent that our approach is able to provide an accurate state density estimation, we extend our NN to do distribution analysis, which is avaluable technique in safety-related applications like autonomous driving.
Here we use an existingreachability tool RPM [21] to compute the forward reachable sets with probability bounds.
Detailsabout how to derive the density and probability bound for the reachable sets are presented in Sec.
3.2and in Appendix B.
Also, RPM was only able to parse the NN for Van der Pol, Double integrator,ground robot navigation, and the FACTEST car model.
It fails in handling other high-dimensioncomplex systems due to numerical issues when partitioning for the input set.
Thus, we only reportthe results on those 4 models in this section.
The main purpose is to show that for some systems the [Figure] density tends to concentrate on certain states, where a small portion of the reachable sets contains themajority of states that are more likely to be reached.
Therefore, our method can better quantify risksthan worst-case reachability, by providing a ﬂexible threshold for the probability of reachability.
We start with the Van der Pol oscillator.
The initial states are uniformly sampled from a squareregion: [ − 2 .
5 , 2 . 5] × [ − 2 . 5 , 2 . 5] .
As illustrated in Fig.
1, the system states will gradually converge toa limit cycle.
Using worst-case reachability analysis for this system will result in a very conservativeover-approximation, and this over-approximation will propagate over time and lead to increasingconservativeness of the reachable set estimation.
As shown in Fig. 3(a), for the worst-case methodslike DryVR(red) [68, 74] and GSG(Green) [73], the volume of their estimated reachable set relativeto the volume of the convex hull of the system states keeps increasing over time, from 1.9670X to5.3027X for the DryVR [68] approach, and from 1.7636X to 2.8086X for GSG [73].
However, ourmethod can give the probability bound for every reachable set in the state space as shown in theheatmap in Fig. 3(b), clearly identifying the region around the limit cycle in high density (brightcolor), and the rest space in low density (dark color).
We can also compute the relative volume of the reachable sets (comparing to the convex hull) preserving different levels of reachable probability, whose evolution over time reﬂects the system’stendency for concentration.
As shown in Fig.
4, we use the above 4 systems and study the volumeof the reachable set with probability threshold 0.50, 0.70, 0.80, 0.90, and 0.99.
As expected, the relative volume will increase as the probability threshold increases.
In all cases, there exist some timeinstances where a small volume of the reachable set actually preserves high probability, which shows the state concentration exists in many existing systems.
While as shown in Fig.
4, the worst-casereachability tools can only generate one curve which presents the (relative) volume of the reachable set that covers all possible states.
Not surprisingly, the worst-case reachability tools give veryconservative results.
We believe using our proposed method to do reachable set distribution analysis will beneﬁt future study for systems with uncertainty, and for systems where the failure case isinevitable but happens with a low probability.
More comparisons with state-of-the-art reachabilitymethods (Verisig [39], Sherlock [75] and ReachNN [40]) are shown in Appendix H.
[Figure]
Online safety veriﬁcation under different initial state distributions.
Since our approachlearns the density concentration function instead of absolute density value, it has the ﬂexibility to estimate online reach sets distributionwith any possible (bounded support) initial distributions.
Consider the safety veriﬁcation forthe ground robot navigation problem (shown in
Fig. 5(a)):
The probability of colliding witha obstacle in the center of the map is determined by the initial state distribution 6 , whichis parametrized as a truncated Gaussian distribution N ( µ, σ 2 I ) where µ and σ measure theexpectation and uncertainty of the initial robot state.
Our method can estimate the upper and lowerbounds for the probability of colliding with the obstacle, and this safety evaluation process can runin faster than 50Hz with parallel computation and heuristic searching used (see details in Appendix
E).
As shown in Fig.
5(b), when the initial state uncertainty σ decreases from 1 . 0 to 0 .
02 , the upper and lower bounds for the probability of collision decrease to close to zero (from 0 .
to3 .
5344 × 10 − 06 , where other worst-case reachability analysis methods can only report a collisionis inevitable, without quantifying the corresponding risks.
This also shows the advantage of our approach in adapting to different density conditions in computation without retraining or ﬁne-tuning.
Limitations and trade-offs of performing reachable set distribution analysis.
Experimentalresults show that our method can compute much less conservative probabilistic reachable sets thanmost worst-case reachability methods.
This less conservative result beneﬁts from RPM which canprovide exact NN reachability analysis by sacriﬁcing scalability.
Therefore RPM also constrains usfrom performing online reachability analysis for high-dimensional systems or systems with large initial sets.
Technically, we are solving a harder problem than worst-case reachability approximation,as we need not only the reachable set, but also the density over those reachable states.
Like worstcase analysis, this is the reason why it can only scale to lower-dimensional systems and smallerinitial sets when we want to perform accurate reachable computation.
We believe that our approachcan better quantify risks under different conditions, especially when unsafe is inevitable (similar to Fig. 5(a)).
Our method can also give worst-case reachability by taking all output reachable cellsproduced by RPM regardless of their density.
This worst-case reachability using RPM is less conservative than other NN reachability tools, at the cost of not scaling to high-dimensional systems.
Conclusion and discussion
In this paper, we propose a Neural Network (NN)-based probabilistic safety veriﬁcation framework that can estimate state density, compute reachable sets and corresponding probability. Our
Liouville-based NN can accurately estimate the state density even for high-dimension systems.
Ourprobabilistic reachable set framework can handle nonlinear (and potentially black-box) systems withvarying initial state distributions and can be used for fast online safety veriﬁcation.
We recognizethat the task of computing probabilistic reachable sets is very useful, and our method is more helpful than worst-case reachability particularly when the system states are more likely to concentrate.
One limitation of our approach is that the NN reachability tool we used (RPM) cannot handle highdimension systems or cases where the initial set is very large, due to the numerical issues whenpartitioning for polyhedral cells.
This limitation is due to the scalability and accuracy trade-off of
NN reachability, which is an independent problem from our paper.
We plan to explore other NNreachability methods and more complicated hybrid systems in real-world applications.