Teaching Temporal Logics to Neural Networks

1 INTRODUCTION . Machine learning has revolutionized several areas of computer science , such as image recognition ( He et al. , 2015 ) , face recognition ( Taigman et al. , 2014 ) , translation ( Wu et al. , 2016 ) , and board games ( Moravcı́k et al. , 2017 ; Silver et al. , 2017 ) . For complex tasks that involve symbolic reasoning , however , deep learning techniques are still considered as insufficient . Applications of deep learning in logical reasoning problems have therefore focused on sub-problems within larger logical frameworks , such as computing heuristics in solvers ( Lederman et al. , 2020 ; Balunovic et al. , 2018 ; Selsam & Bjørner , 2019 ) or predicting individual proof steps ( Loos et al. , 2017 ; Gauthier et al. , 2018 ; Bansal et al. , 2019 ; Huang et al. , 2018 ) . Recently , however , the assumption that deep learning is not yet ready to tackle hard logical questions was drawn into question . Lample & Charton ( 2020 ) demonstrated that Transformer models ( Vaswani et al. , 2017 ) perform surprisingly well on symbolic integration , Rabe et al . ( 2020 ) demonstrated that self-supervised training leads to mathematical reasoning abilities , and Brown et al . ( 2020 ) demonstrated that large-enough language models learn basic arithmetic despite being trained on mostly natural language sources . This poses the question if other problems that are thought to require symbolic reasoning lend themselves to a direct learning approach . We study the application of Transformer models to challenging ∗Partially supported by the European Research Council ( ERC ) Grant OSARES ( No . 683300 ) and the Collaborative Research Center “ Foundations of Perspicuous Software Systems ” ( TRR 248 , 389792660 ) . logical problems in verification . We thus consider linear-time temporal logic ( LTL ) ( Pnueli , 1977 ) , which is widely used in the academic verification community ( Dwyer et al. , 1998 ; Li et al. , 2013 ; Duret-Lutz et al. , 2016 ; Rozier & Vardi , 2007 ; Schuppan & Darmawan , 2011 ; Li et al. , 2013 ; 2014 ; Schwendimann , 1998 ) and is the basis for industrial hardware specification languages like the IEEE standard PSL ( IEEE-Commission et al. , 2005 ) . LTL specifies infinite sequences and is typically used to describe system behaviors . For example , LTL can specify that some proposition P must hold at every point in time ( P ) or that P must hold at some future point of time ( P ) . By combining these operators , one can specify that P must occur infinitely often ( P ) . In this work , we apply a direct learning approach to the fundamental problem of LTL to find a satisfying trace to a formula . In applications , solutions to LTL formulas can represent ( counter ) examples for a specified system behavior , and over the last decades , generations of advanced algorithms have been developed to solve this question automatically . We start from the standard benchmark distribution of LTL formulas , consisting of conjunctions of patterns typically encountered in practice ( Dwyer et al. , 1998 ) . We then use classical algorithms , notably spot by Duret-Lutz et al . ( 2016 ) , that implement a competitive classical algorithm , to generate solutions to formulas from this distribution and train a Transformer model to predict these solutions directly . Relatively small Transformers perform very well on this task and we predict correct solutions to 96.8 % of the formulas from a held-out test set ( see Figure 1 ) . Impressive enough , Transformers hold up pretty well and predict correct solutions in 83 % of the cases , even when we focus on formulas on which spot timed out . This means that , already today , direct machine learning approaches may be useful to augment classical algorithms in logical reasoning tasks . We also study two generalization properties of the Transformer architecture , important to logical problems : We present detailed analyses on the generalization to longer formulas . It turns out that transformers trained with tree-positional encodings ( Shiv & Quirk , 2019 ) generalize to much longer formulas than they were trained on , while Transformers trained with the standard positional encoding ( as expected ) do not generalize to longer formulas . The second generalization property studied here is the question whether Transformers learn to imitate the generator of the training data , or whether they learn to solve the formulas according to the semantics of the logics . This is possible , as for most formulas there are many possible satisfying traces . In Figure 1 we highlight the fact that our models often predicted traces that satisfy the formulas , but predict different traces than the one found by the classical algorithm with which we generated the data . Especially when testing the models out-of-distribution we observed that almost no predicted trace equals the solution proposed by the classical solver . To demonstrate that these generalization behaviors are not specific to the benchmark set of LTL formulas , we also present experimental results on random LTL formulas . Further , we exclude that spot , the tool with which we generate example traces , is responsible for these behaviors , by repeating the experiments on propositional formulas for which we generate the solutions by SAT solvers . The remainder of this paper is structured as follows . We give an overview over related work in Section 2 . We describe the problem definitions and present our data generation in Section 3 . Our experimental setup is described in Section 4 and our findings in Section 5 , before concluding in Section 6 . 2 RELATED WORK . Datasets for mathematical reasoning . While we focus on a classical task from verification , other works have studied datasets derived from automated theorem provers ( Blanchette et al. , 2016 ; Loos et al. , 2017 ; Gauthier et al. , 2018 ) , interactive theorem provers ( Kaliszyk et al. , 2017 ; Bansal et al. , 2019 ; Huang et al. , 2018 ; Yang & Deng , 2019 ; Polu & Sutskever , 2020 ; Wu et al. , 2020 ; Li et al. , 2020 ; Lee et al. , 2020 ; Urban & Jakubův , 2020 ; Rabe et al. , 2020 ) , symbolic mathematics ( Lample & Charton , 2020 ) , and mathematical problems in natural language ( Saxton et al. , 2019 ; Schlag et al. , 2019 ) . Probably the closest work to this paper are the applications of Transformers to directly solve differential equations ( Lample & Charton , 2020 ) and directly predict missing assumptions and types of formal mathematical statements ( Rabe et al. , 2020 ) . We focus on a different problem domain , verification , and demonstrate that Transformers are roughly competitive with classical algorithms in that domain on their dataset . Learning has been applied to mathematics long before the rise of deep learning . Earlier works focused on ranking premises or clauses Cairns ( 2004 ) ; Urban ( 2004 ; 2007 ) ; Urban et al . ( 2008 ) ; Meng & Paulson ( 2009 ) ; Schulz ( 2013 ) ; Kaliszyk & Urban ( 2014 ) . Neural architectures for logical reasoning . ( Paliwal et al. , 2020 ) demonstrate significant improvements in theorem proving through the use of graph neural networks to represent higher-order logic terms . Selsam et al . ( 2019 ) presented NeuroSAT , a graph neural network ( Scarselli et al. , 2008 ; Li et al. , 2017 ; Gilmer et al. , 2017 ; Wu et al. , 2019 ) for solving the propositional satisfiability problem . In contrast , we apply a generic sequence-to-sequence model to predict the solutions to formulas , not only whether there is a solution . This allows us to apply the approach to a wider set of logics ( logics without a CNF ) . A simplified NeuroSAT architecture was trained for unsat-core predictions ( Selsam & Bjørner , 2019 ) . Lederman et al . ( 2020 ) have used graph neural networks on CNF to learn better heuristics for a 2QBF solver . Evans et al . ( 2018 ) study the problem of logical entailment in propositional logic using tree-RNNs . Entailment is a subproblem of satisfiability and ( besides being a classification problem ) could be encoded in the same form as our propositional formulas . The formulas considered in their dataset are much smaller than in this work . Language models applied to programs . Transformers have also been applied to programs for tasks such as summarizing code ( Fernandes et al. , 2018 ) or variable naming and misuse ( Hellendoorn et al. , 2020 ) . Other works focused on recurrent neural networks or graph neural networks for code analysis , e.g . ( Piech et al. , 2015 ; Gupta et al. , 2017 ; Bhatia et al. , 2018 ; Wang et al. , 2018 ; Allamanis et al. , 2017 ) . Another area in the intersection of formal methods and machine learning is the verification of neural networks ( Seshia & Sadigh , 2016 ; Seshia et al. , 2018 ; Singh et al. , 2019 ; Gehr et al. , 2018 ; Huang et al. , 2017 ; Dreossi et al. , 2019 ) . 3 DATASETS . To demonstrate the generalization properties of the Transformer on logical tasks , we generated several datasets in three different fashions . We will describe the underlying logical problems and our data generation in the following . 3.1 TRACE GENERATION FOR LINEAR-TIME TEMPORAL LOGIC . Linear-time temporal logic ( LTL , Pnueli , 1977 ) combines propositional connectives with temporal operators such as the Next operator and the Until operator U . ϕ means that ϕ holds in the next position of a sequence ; ϕ1 U ϕ2 means that ϕ1 holds until ϕ2 holds . For example , the LTL formula ( bU a ) ∧ ( cU ¬a ) states that b has to hold along the trace until a holds and c has to hold until a does not hold anymore . There also exist derived operators . For example , consider the following specification of an arbiter : ( request → grant ) states that , at every point in time ( -operator ) , if there is a request signal , then a grant signal must follow at some future point in time ( -operator ) . The full semantics and an explanation of the operators can be found in Appendix A . We consider infinite sequences , that are finitely represented in the form of a “ lasso ” uvω , where u , called prefix , and v , called period , are finite sequences of propositional formulas . We call such sequences ( symbolic ) traces . For example , the symbolic trace ( a ∧ b ) ω defines the infinite sequence where a and b evaluate to true on every position . Symbolic traces allow us to underspecify propositions when they do not matter . For example , the LTL formula a is satisfied by the symbolic trace : true true ( a ) ω , which allow for any combination of propositions on the first two positions . Our datasets consist of pairs of satisfiable LTL formulas and satisfying symbolic traces generated with tools and automata constructions from the spot framework ( Duret-Lutz et al. , 2016 ) . We use a compact syntax for ultimately periodic symbolic traces : Each position in the trace is separated by the delimiter “ ; ” . True and False are represented by “ 1 ” and “ 0 ” , respectively . The beginning of the period v is signaled by the character “ { ” and analogously its end by “ } ” . For example , the ultimately periodic symbolic trace denoted by a ; a ; a ; { b } , describes all infinite traces where on the first 3 positions a must hold followed by an infinite period on which b must hold on every position . Given a satisfiable LTL formula ϕ , our trace generator constructs a Büchi automatonAϕ that accepts exactly the language defined by the LTL formula , i.e. , L ( Aϕ ) = L ( ϕ ) . From this automaton , we construct an arbitrary accepted symbolic trace , by searching for an accepting run in Aϕ .

Summary: The paper presents multiple dataset generation and testing procedures for linear temporal logic and propositional logic satisfiability. They are then used to train Transformers with tree positional encoding. The approach amounts to imitation learning based on existing solvers for satisfiability for the considered logics. In contrast to previous work, the approach supports logical formulas of arbitrary shape. The experiments demonstrate successful generalization for multiple approaches to dataset generation.