Safe Neurosymbolic Learning with Differentiable Symbolic Execution

We study the problem of learning worst-case-safe parameters for programs that use neural networks as well as symbolic , human-written code . Such neurosymbolic programs arise in many safety-critical domains . However , because they need not be continuous , let alone differentiable , they can not be learned using existing gradient-based approaches to safe learning . Our method , Differentiable Symbolic Execution ( DSE ) , learns such programs by sampling code paths using symbolic execution , constructing gradients of a worst-case “ safety loss ” along these paths , and then backpropagating these gradients through program operations using a generalization of the REINFORCE estimator . We evaluate the method on a mix of synthetic tasks and real-world benchmarks . Our experiments show that DSE significantly outperforms the state-of-the-art DIFFAI method on these tasks . 1 INTRODUCTION . Safety on worst-case inputs has recently emerged as a key challenge in deep learning research . Formal verification of neural networks ( Albarghouthi , 2021 ) is an established response to this challenge . In particular , an exciting body of recent work ( Mirman et al. , 2018 ; Madry et al. , 2017 ; Cohen et al. , 2019 ; Singh et al. , 2018 ) has sought to incorporate formal verification into the training of neural networks . DIFFAI , among the most prominent of such approaches , uses a neural network verifier to construct a differentiable , worst-case safety loss for the learner . This loss is used to regularize a standard data-driven loss , biasing the learner towards parameters that are both performant and safe . A weakness of these methods is that they only consider functional properties ( such as adversarial robustness ) of isolated neural networks . By contrast , in real-world applications , neural networks are often embedded within human-written symbolic code , and correctness requirements apply to the entire neurosymbolic composition . For example , consider a car directed by a neural controller ( Qin & Badgwell , 2000 ) . Safety properties for the car are functions of its trajectories , and these trajectories depend not just on the controller but also the symbolic equations that define the environment . While recent work ( Christakis et al. , 2021 ) has studied the verification of such neurosymbolic programs , there is no prior work on integrating verification and learning for such systems . In this paper , we present the first steps towards such an integration . The fundamental technical challenge here is that while a neural network is differentiable over its parameters , the code surrounding may be non-differentiable or even discontinuous . This puts our problem beyond the scope of existing methods for integrated learning and verification . We overcome this difficulty using a new method , Differentiable Symbolic Execution ( DSE ) , for estimating gradients of worst-case safety losses of nondifferentiable neurosymbolic programs . DSE is based on a generalization of the classic REINFORCE estimator , which backpropagates gradients through non-differentiable operations by approximating integrals with sampling . In our problem , the integral is the aggregate of the safety losses along symbolic control flow paths in the program . To apply REINFORCE-like ideas here , we need to represent and sample paths in a symbolic way . We do so through an adaptation of the classic method of symbolic execution . We evaluate DSE through several case studies in the embedded control and navigation domains . Our baselines include an extended version of DIFFAI , the current state of the art , and an ablation that does not use an explicit safety loss . Our experiments show that DSE significantly outperforms the baselines in finding safe and high-performance model parameters . We also do a series of synthetic experiments that highlight the practical difficulty that DIFFAI-style approaches face in our setting . To summarize , our main contributions are : • We present the first approach to worst-case-safe parameter learning for neural networks embedded within nondifferentiable , symbolic programs . • As part of our learning algorithm , we give a new way to bring together symbolic execution and stochastic gradient estimators that might have applications outside our immediate task . • We present experimental results that indicate the advantages of DSE over the state-of-the-art in verified learning . 2 PROBLEM FORMULATION . Programs . We define programs with embedded neural networks as symbolic transition systems ( STS ) ( Manna & Pnueli , 2012 ) . Formally , a program Fθ is a tuple ( Loc , X , l0 , Init , Safe , Transθ ) . Here , Loc is a finite set of ( control ) locations , X = { x1 , . . . , xm } is a set of real-valued variables , and l0 ∈ Loc is the initial location . Init , a boolean formula over X , is an initial condition for the program . Safe is a map from locations to constraints over X ; intuitively , Safe ( l ) is a safety requirement asserted at location l. Finally , Transθ is a transition relation consisting of transitions ( l , G , Uθ , l ′ ) such that : ( i ) l is the source location and l′ is the destination location ; ( ii ) the guard G is a constraint over X ; and ( iii ) the update Uθ is a vector 〈U1 , θ , . . . , Um , θ〉 , where each Ui , θ is a real-valued expression over X constructed using standard symbolic operators and neural networks with parameters θ . Intuitively , Ui , θ represents the update to the i-th variable . We assume that each Ui , θ is differentiable in θ . Also , we assume that the programs are deterministic . That is , if G and G′ are guards for two distinct transitions from the same source state , then G ∧G′ is unsatisfiable . Programs in higher-level languages can be translated to the STS notation in a standard way . For example , Figure 1 ( left ) shows a simple high-level program . The STS for this program appears in Figure 1 ( right ) . Remarkably , while the program is simple , the state-of-the-art DIFFAI approach to verified learning fails to learn safe parameters for it . Safety Semantics . In classical formal methods , a program is considered to be safe if all of its executions satisfy a logical constraint . However , in learning settings , it helps to know not only whether a program is unsafe , but also the extent to which it is unsafe . Consequently , we define the safety semantics of programs in a quantitative way , in terms of a ( worst-case ) safety loss that quantifies the extent program ’ s safeness . Formally , let a state of Fθ be a pair s = ( l , v ) , where l is a location and v ∈ Rm is an assignment of values to the variables ( i.e. , v ( i ) is the value of xi ) . Such a state is said to be at location l. For boolean constraints B and assignments v to variables in B , let us write B ( v ) if v satisfies B . A state ( l0 , v ) , where Init ( v ) , is an initial state . A state ( l , v ) is safe if ( Safe ( l ) ) ( v ) . Let v ∈ Rm be an assignment to the variables . For a real-valued expression E over X , let E ( v ) be the value of E when xi is substituted by v ( i ) . For an update U = 〈U1 , . . . , Un〉 , we define U ( v ) as the assignment 〈U1 ( v ) , . . . , Un ( v ) 〉 . A length-n trajectory of Fθ is a sequence τ = 〈s0 , . . . , sn〉 , with si = ( li , vi ) , such that : ( i ) s0 is an initial state ; and ( ii ) for each i , there is a transition ( li , G , U , li+1 ) such that G ( vi ) and vi+1 = U ( vi ) . Let us fix a size bound N for trajectories . A trajectory τ = 〈s0 , . . . , sn〉 is maximal if it has length N , or if there is no trajectory τ ′ = 〈s0 , . . . , sn , sn+1〉 with length ≤ N . Because our programs are deterministic , there is a unique maximal trajectory from each s0 . We denote this trajectory by τ ( s0 ) . Let us assume a real-valued loss Unsafe ( s ) that quantifies the unsafeness of each state s. We require Unsafe ( s ) = 0 if s is safe and Unsafe ( s ) > 0 otherwise . We lift this measure to trajectories τ by letting Unsafe ( τ ) = ∑ s appears in τ Unsafe ( s ) . The safety loss C ( θ ) for Fθ is now defined as : C ( θ ) = max s is an initial state Unsafe ( τ ( s ) ) . ( 1 ) Thus , C ( θ ) = 0 if and only if all program trajectories are safe . Problem Statement . Our learning problem formalizes a setting in which we have training data for neural networks inside a program Fθ . While training the networks with respect to this data , we must ensure that the overall program satisfies its safety requirements . To ensure that the parameters of the different neural networks in Fθ are not incorrectly entangled , we assume that only one of these networks , NNθ , has trainable parameters.1 We expect as input a training set of i.i.d . samples from an unknown distribution over the inputs and outputs of NNθ , and a differentiable data loss Q ( θ ) that quantifies the network ’ s fidelity to this training set . Our learning goal is to solve the following constrained optimization problem : min θ Q ( θ ) s.t . C ( θ ) ≤ 0 . ( 2 ) 3 LEARNING ALGORITHM . Our learning approach is based on two ideas . First , we directly apply a recently-developed equivalence between constrained and regularized learning ( Agarwal et al. , 2018 ; Le et al. , 2019 ) to reduce Equation ( 2 ) to a series of unconstrained optimization problems . Second , we use the novel technique of Differentiable Symbolic Execution ( DSE ) to solve these unconstrained problems . At the highest level , we convexify the class of programs Fθ by allowing stochastic combination of programs . That is , we now allow programs of the form Fθ̂ = ∑T t=1 αtFθt , where ∑ i αt = 1 . To execute the program Fθ from a given initial state , one samples a specific program Fθt from the distribution ( α1 , . . . , αt ) and then executes it from that state . Equation ( 2 ) can now be written into the problem maxλ∈R+ minθ̂ Q ( θ̂ ) + λC ( θ̂ ) , which in turn can be solved using a classic algorithm ( Freund & Schapire , 1999 ) for computing equilibria in a twoplayer game . We omit further details about this algorithm because it follows Le et al . ( 2019 ) . ( For more details , see Appendix A.1 ) . A key feature of our high-level algorithm is that it repeatedly solves the optimization problem min θ Q ( θ ) + λC ( θ ) ( 3 ) for fixed values of λ . This problem is challenging because while Q ( θ ) is differentiable in θ , C ( θ ) depends on the entirety of Fθ and may not even be continuous . As we demonstrate in Section 5 , this makes it difficult to apply state-of-the-art gradient-based approaches to worst-case safe learning . DSE , our main contribution , addresses this challenge by estimating gradients ∇θC # ( θ ) of a differentiable approximation C # ( θ ) of C ( θ ) . We present the details of this method in the next section .

Summary: This paper presents an approach to learn worst-case safe parameters for neurosymbolic programs (programs with neural networks + symbolic portions). The key contribution is an algorithm to compute gradients for the worst-case safety loss by symbolically sampling control paths in the programs and using a modification of the Reinforce estimate to approximate the gradients. The approach is compared with DiffAI on several synthetic tasks.