DL2: Training and Querying Neural Networks with Logic

1 INTRODUCTION . With the success of neural networks across a wide range of important application domains , a key challenge that has emerged is that of making neural networks more reliable . Promising directions to address this challenge are incorporating constraints during training ( Madry et al. , 2017 ; Minervini et al. , 2017 ) and inspecting already trained networks by posing specific queries ( Goodfellow et al. , 2014b ; Pei et al. , 2017 ; Xu et al. , 2018 ) ) . While useful , these approaches are described and hardcoded to particular kinds of constraints , making their application to other settings difficult . Inspired by prior work ( e.g. , Cohen et al . ( 2017 ) ; Fu & Su ( 2016 ) ; Hu et al . ( 2016 ) ; Bach et al . ( 2017 ) ) , we introduce a new method and system , called DL2 ( acronym for Deep Learning with Differentiable Logic ) , which can be used to : ( i ) query networks for inputs meeting constraints , and ( ii ) train networks to meet logical specifications , all in a declarative fashion . Our constraint language can express rich combinations of arithmetic comparisons over inputs , neurons and outputs of neural networks using negations , conjunctions , and disjunctions . Thanks to its expressiveness , DL2 enables users to enforce domain knowledge during training or interact with the network in order to learn about its behavior via querying . DL2 works by translating logical constraints into non-negative loss functions with two key properties : ( P1 ) a value where the loss is zero is guaranteed to satisfy the constraints , and ( P2 ) the resulting loss is differentiable almost everywhere . Combined , these properties enable us to solve the problem of querying or training with constraints by minimizing a loss with off-the-shelf optimizers . Training with DL2 To make optimization tractable , we exclude constraints on inputs that capture convex sets and include them as constraints to the optimization goal . We then optimize with projected gradient descent ( PGD ) , shown successful for training with robustness constraints ( Madry et al. , 2017 ) . The expressiveness of DL2 along with tractable optimization through PGD enables us to train with new , interesting constraints . For example , we can express constraints over probabilities which are not explicitly computed by the network . Consider the following : ∀x . pθpeople ( x ) < ∨ pθpeople ( x ) > 1− This constraint , in the context of CIFAR-100 , says that for any network input x ( network is parameterized by θ ) , the probability of people ( ppeople ) is either very small or very large . However , CIFAR-100 does not have the class people , and thus we define it as a function of other probabilities , in particular : ppeople = pbaby + pboy + pgirl + pman + pwoman . We show that with a similar constraint ( but with 20 classes ) , DL2 increases the prediction accuracy of CIFAR-100 networks in the semi-supervised setting , outperforming prior work whose expressiveness is more restricted . DL2 can capture constraints arising in both , classification and regression tasks . For example , GalaxyGAN ( Schawinski et al. , 2017 ) , a generator of galaxy images , requires the network to respect constraints imposed by the underlying physical systems , e.g. , flux : the sum of input pixels should equal the sum of output pixels . Instead of hardcoding such a constraint into the network in an ad hoc way , with DL2 , this can now be expressed declaratively : sum ( x ) = sum ( GalaxyGAN ( x ) ) . Global training A prominent feature of DL2 is its ability to train with constraints that place restrictions on inputs outside the training set . Prior work on training with constraints ( e.g. , Xu et al . ( 2018 ) ) focus on the given training set to locally train the network to meet the constraints . With DL2 , we can , for the first time , query for inputs which are outside the training set , and use them to globally train the network . Previous methods that trained on examples outside the training set were either tailored to a specific task ( Madry et al. , 2017 ) or types of networks ( Minervini et al. , 2017 ) . Our approach splits the task of global training between : ( i ) the optimizer , which trains the network to meet the constraints for the given inputs , and ( ii ) the oracle , which provides the optimizer with new inputs that aim to violate the constraints . To illustrate , consider the following Lipshcitz condition : ∀z1 ∈ L∞ ( x1 , ) , z2 ∈ L∞ ( x2 , ) .||pθ ( z1 ) − pθ ( z2 ) ||2 < L||z1 − z2||2 Here , for two inputs from the training set ( x1 , x2 ) , any point in their -neighborhood ( z1 , z2 ) must satisfy the condition . This constraint is inspired by recent works ( e.g. , Gouk et al . ( 2018 ) ; Balan et al . ( 2017 ) ) which showed that neural networks are more stable if satisfying the Lipschitz condition . Querying with DL2 We also designed an SQL-like language which enables users to interact with the model by posing declarative queries . For example , consider the scenarios studied by a recent work ( Song et al. , 2018 ) where authors show how to generate adversarial examples with ACGANs ( Odena et al. , 2016 ) . The generator is used to create images from a certain class ( e.g. , 1 ) which fools a classifier ( to classify as , e.g. , 7 ) . With DL2 , this can be phrased as : f i n d n [ 100 ] where n in [ -1 , 1 ] , c l a s s ( M_NN1 ( M_ACGAN_G ( n , 1 ) ) ) = 7 re turn M_ACGAN_G ( n , 1 ) This query aims to find an input n ∈ R100 to the generator satisfying two constraints : its entries are between−1 and 1 ( enforcing a domain constraint ) and it results in the generator producing an image , which it believes to be classified as 1 ( enforced by M_ACGAN_G ( n , 1 ) ) but is classified by the network ( M_NN1 ) as 7 . DL2 automatically translates this query to a DL2 loss and optimizes it with an off-the-shelf optimizer ( L-BFGS-B ) to find solutions , in this case , the image to the right . Our language can naturally capture many prior works at the declarative level , including finding neurons responsible for a given prediction ( Olah et al. , 2018 ) , inputs that differentiate two networks ( Pei et al. , 2017 ) , and adversarial example generation ( e.g. , Szegedy et al . ( 2013 ) ) . Main Contributions The DL2 system is based on the following contributions : • An approach for training and querying neural networks with logical constraints based on translating these into a differentiable loss with desirable properties ( ( P1 ) and ( P2 ) ) . • A training procedure which extracts constraints on inputs that capture convex sets and includes them as PGD constraints , making optimization tractable . • A declarative language for posing queries over neural network ’ s inputs , outputs , and internal neurons . Queries are compiled into a differentiable loss and optimized with L-BFGS-B . • An extensive evaluation demonstrating the effectiveness of DL2 in querying and training neural networks . Among other experimental results , we show for the first time , the ability to successfully train networks with constraints on inputs not in the training set . 2 RELATED WORK . Adversarial example generation ( Pei et al. , 2017 ; Goodfellow et al. , 2014b ) can be seen as a fixed query to the network , while adversarial training ( Madry et al. , 2017 ) aims to enforce a specific constraint . Most works aiming to train networks with logic impose soft constraints , often through an additional loss ( Pathak et al. , 2015 ; Xu et al. , 2018 ) ; ( Márquez-Neila et al. , 2017 ) shows that hard constraints have no empirical advantage over soft constraints . Probabilistic Soft Logic ( PSL ) ( Kimmig et al. , 2012 ) translates logic into continuous functions over [ 0 , 1 ] . As we show , PSL is not amenable to gradient-based optimization as gradients may easily become zero . Hu et al . ( 2016 ) builds on PSL and presents a teacher-student framework which distills rules into the training phase . The idea is to formulate rule satisfaction as a convex problem with a closed-form solution . However , this formulation is restricted to rules over random variables and can not express rules over probability distributions . In contrast , DL2 can express such constraints , e.g. , p1 > p2 , which requires the network probability for class 1 is greater than for 2 . Also , the convexity and the closed-form solution stem from the linearity of the rules in the network ’ s output , meaning that non-linear constraints ( e.g. , Lipschitz condition , expressible with DL2 ) are fundamentally beyond the reach of this method . The work of Xu et al . ( 2018 ) is also restricted to constraints over random variables and is intractable for complicated constraints . Fu & Su ( 2016 ) reduces the satisfiability of floating-point formulas into numerical optimization , however , their loss is not differentiable and they do not support constraints on distributions . Finally , unlike DL2 , no prior work supports constraints for regression tasks . 3 FROM LOGIC TO A DIFFERENTIABLE LOSS . We now present our constraint language and show how to translate constraints into a differentiable loss . To simplify presentation , we treat all tensors as vectors with matching dimensions . Logical Language Our language consists of quantifier-free constraints which can be formed with conjunction ( ∧ ) , disjunction ( ∨ ) and negation ( ¬ ) . Atomic constraints ( literals ) are comparisons ./ of terms ( here ./ ∈ { = , 6= , ≤ , < , ≥ , > } ) . Comparisons are defined for scalars and applied elementwise on vectors . A term t is : ( i ) A variable z or a constant c , representing real-valued vectors ; constants can be samples from the dataset . ( ii ) An expression over terms , including arithmetic expressions or function applications f : Rm → Rn , for m , n ∈ Z+ . Functions can be defined overvariables , constants , and network parameters θ1 , . . . , θl . Functions can be the application of a network with parameters θ , the application of a specific neuron , or a computation over multiple networks . The only assumption on functions is that they are differentiable ( almost everywhere ) in the variables and network parameters . We write t ( z1 , . . . , zk , c1 , . . . , cj , θ1 , . . . θl ) to emphasize the variables , constants , and network parameters that t can be defined over ( that is , t may refer to only a subset of these symbols ) . We sometimes omit the constants and network parameters ( which are also constant ) and abbreviate variables by z̄ , i.e. , we write t ( z̄ ) . Similarly , we write ϕ ( z̄ ) to denote a constraint defined over variables z̄ . When variables are not important , we write ϕ . Translation into loss Given a formula ϕ , we define the corresponding loss L ( ϕ ) recursively on the structure of ϕ . The obtained loss is non-negative : for any assignment x̄ to the variables z̄ , we have L ( ϕ ) ( x̄ ) ∈ R≥0 . Further , the translation has two properties : ( P1 ) any x̄ for which the loss is zero ( L ( ϕ ) ( x̄ ) = 0 ) is a satisfying assignment to ϕ ( denoted by x̄ |= ϕ ) and ( P2 ) the loss is differentiable almost everywhere . This construction avoids pitfalls of other approaches ( see Appendix B ) . We next formally define the translation rules . Formula ϕ is parametrized by ξ > 0 which denotes tolerance for strict inequality constraints . Since comparisons are applied element-wise ( i.e. , on scalars ) , atomic constraints are transformed into a conjunction of scalar comparisons : L ( t1 ./ t2 ) : = L ( n∧ i=1 t1i ./ t 2 i ) The comparisons = and ≤ are translated based on a function d : R×R→ R which is a continuous , differentiable almost everywhere , distance function with d ( x1 , x2 ) ≥0 and d ( x1 , x2 ) =0↔ x1=x2 : L ( t1 = t2 ) : = d ( t1 , t2 ) ; L ( t1 ≤ t2 ) : = 1t1 > t2 · d ( t1 , t2 ) Here , 1t1 > t2 is an indicator function : it is 1 if t1 > t2 , and 0 otherwise . The function d is a parameter of the translation and in our implementation we use the absolute distance |t1 − t2| . For the other comparisons , we define the loss as follows : L ( t1 < t2 ) = L ( t1 + ξ ≤ t2 ) , L ( t1 6= t2 ) = L ( t1 < t2 ∨ t2 < t1 ) , and L ( t1 > t2 ) and L ( t1 ≥ t2 ) are defined analogously . Conjunctions and disjunctions of formulas ϕ and ψ are translated into loss as follows : L ( ϕ ∧ ψ ) : = L ( ϕ ) + L ( ψ ) ; L ( ϕ ∨ ψ ) : = L ( ϕ ) · L ( ψ ) Note that L ( ϕ ∧ ψ ) = 0 if and only if L ( ϕ ) = 0 and L ( ψ ) = 0 , which by construction is true if ϕ and ψ are satisfied , and similarly L ( ϕ ∨ ψ ) = 0 if and only if L ( ϕ ) = 0 or L ( ψ ) = 0 . Translating negations Negations are handled by first eliminating them from the constraint through rewrite rules , and then computing the loss of their equivalent , negation-free constraint . Negations of atomic constraints are rewritten to an equivalent atomic constraint that has no negation ( note that 6= is not a negation ) . For example , the constraint ¬ ( t1 ≤ t2 ) is rewritten to t2 < t1 , while negations of conjunctions and disjunctions are rewritten by repeatedly applying De Morgan ’ s laws : ¬ ( ϕ ∧ ψ ) is equivalent to ¬ϕ ∨ ¬ψ and ¬ ( ϕ ∨ ψ ) is equivalent to ¬ϕ ∧ ¬ψ . With our construction , we get the following theorem : Theorem 1 For all x̄ , if L ( ϕ ) ( x̄ ) = 0 , then x̄ satisfies ϕ ( x̄ |= ϕ ) . Conversely , for any ϕ , there is a δ ( ξ ) ≥ 0 with limξ→0 δ ( ξ ) = 0 such that for all x̄ , L ( ϕ ) ( x̄ ) > δ ( ξ ) implies that x̄ does not satisfy ϕ ( x̄ 6|= ϕ ) . Essentially , as we make ξ smaller ( and δ approaches 0 ) , we get closer to an if and only if theorem : if x̄ makes the loss 0 , then we have a satisfying assignment ; otherwise , if x̄ makes the loss > δ , then x̄ is not a satisfying assignment . We provide the proof of the theorem in Appendix A .

Summary: This paper proposes DL2, a framework for turning queries over parameters and input, output pairs to neural networks into differentiable loss functions, and an associated declarative language for specifying these queries. The motivation for this work is twofold. The first is to allow for the specification of additional domain knowledge during training. For example, if a user expects that the predicted probabilities of some output classes should be correlated for all predictions, this constraint can be enforced during weight learning. Second, it allows users to search for specific inputs that satisfy specified conditions. In this way, DL2 can capture popular applications like searching for adversarial examples by querying for inputs close to a known input of class A but that the network predicts is class B with high confidence.