Teaching Temporal Logics to Neural Networks

1 INTRODUCTION . Machine learning has revolutionized several areas of computer science , such as image recognition ( He et al. , 2015 ) , face recognition ( Taigman et al. , 2014 ) , translation ( Wu et al. , 2016 ) , and board games ( Moravcı́k et al. , 2017 ; Silver et al. , 2017 ) . For complex tasks that involve symbolic reasoning , however , deep learning techniques are still considered as insufficient . Applications of deep learning in logical reasoning problems have therefore focused on sub-problems within larger logical frameworks , such as computing heuristics in solvers ( Lederman et al. , 2020 ; Balunovic et al. , 2018 ; Selsam & Bjørner , 2019 ) or predicting individual proof steps ( Loos et al. , 2017 ; Gauthier et al. , 2018 ; Bansal et al. , 2019 ; Huang et al. , 2018 ) . Recently , however , the assumption that deep learning is not yet ready to tackle hard logical questions was drawn into question . Lample & Charton ( 2020 ) demonstrated that Transformer models ( Vaswani et al. , 2017 ) perform surprisingly well on symbolic integration , Rabe et al . ( 2020 ) demonstrated that self-supervised training leads to mathematical reasoning abilities , and Brown et al . ( 2020 ) demonstrated that large-enough language models learn basic arithmetic despite being trained on mostly natural language sources . This poses the question if other problems that are thought to require symbolic reasoning lend themselves to a direct learning approach . We study the application of Transformer models to challenging ∗Partially supported by the European Research Council ( ERC ) Grant OSARES ( No . 683300 ) and the Collaborative Research Center “ Foundations of Perspicuous Software Systems ” ( TRR 248 , 389792660 ) . logical problems in verification . We thus consider linear-time temporal logic ( LTL ) ( Pnueli , 1977 ) , which is widely used in the academic verification community ( Dwyer et al. , 1998 ; Li et al. , 2013 ; Duret-Lutz et al. , 2016 ; Rozier & Vardi , 2007 ; Schuppan & Darmawan , 2011 ; Li et al. , 2013 ; 2014 ; Schwendimann , 1998 ) and is the basis for industrial hardware specification languages like the IEEE standard PSL ( IEEE-Commission et al. , 2005 ) . LTL specifies infinite sequences and is typically used to describe system behaviors . For example , LTL can specify that some proposition P must hold at every point in time ( P ) or that P must hold at some future point of time ( P ) . By combining these operators , one can specify that P must occur infinitely often ( P ) . In this work , we apply a direct learning approach to the fundamental problem of LTL to find a satisfying trace to a formula . In applications , solutions to LTL formulas can represent ( counter ) examples for a specified system behavior , and over the last decades , generations of advanced algorithms have been developed to solve this question automatically . We start from the standard benchmark distribution of LTL formulas , consisting of conjunctions of patterns typically encountered in practice ( Dwyer et al. , 1998 ) . We then use classical algorithms , notably spot by Duret-Lutz et al . ( 2016 ) , that implement a competitive classical algorithm , to generate solutions to formulas from this distribution and train a Transformer model to predict these solutions directly . Relatively small Transformers perform very well on this task and we predict correct solutions to 96.8 % of the formulas from a held-out test set ( see Figure 1 ) . Impressive enough , Transformers hold up pretty well and predict correct solutions in 83 % of the cases , even when we focus on formulas on which spot timed out . This means that , already today , direct machine learning approaches may be useful to augment classical algorithms in logical reasoning tasks . We also study two generalization properties of the Transformer architecture , important to logical problems : We present detailed analyses on the generalization to longer formulas . It turns out that transformers trained with tree-positional encodings ( Shiv & Quirk , 2019 ) generalize to much longer formulas than they were trained on , while Transformers trained with the standard positional encoding ( as expected ) do not generalize to longer formulas . The second generalization property studied here is the question whether Transformers learn to imitate the generator of the training data , or whether they learn to solve the formulas according to the semantics of the logics . This is possible , as for most formulas there are many possible satisfying traces . In Figure 1 we highlight the fact that our models often predicted traces that satisfy the formulas , but predict different traces than the one found by the classical algorithm with which we generated the data . Especially when testing the models out-of-distribution we observed that almost no predicted trace equals the solution proposed by the classical solver . To demonstrate that these generalization behaviors are not specific to the benchmark set of LTL formulas , we also present experimental results on random LTL formulas . Further , we exclude that spot , the tool with which we generate example traces , is responsible for these behaviors , by repeating the experiments on propositional formulas for which we generate the solutions by SAT solvers . The remainder of this paper is structured as follows . We give an overview over related work in Section 2 . We describe the problem definitions and present our data generation in Section 3 . Our experimental setup is described in Section 4 and our findings in Section 5 , before concluding in Section 6 . 2 RELATED WORK . Datasets for mathematical reasoning . While we focus on a classical task from verification , other works have studied datasets derived from automated theorem provers ( Blanchette et al. , 2016 ; Loos et al. , 2017 ; Gauthier et al. , 2018 ) , interactive theorem provers ( Kaliszyk et al. , 2017 ; Bansal et al. , 2019 ; Huang et al. , 2018 ; Yang & Deng , 2019 ; Polu & Sutskever , 2020 ; Wu et al. , 2020 ; Li et al. , 2020 ; Lee et al. , 2020 ; Urban & Jakubův , 2020 ; Rabe et al. , 2020 ) , symbolic mathematics ( Lample & Charton , 2020 ) , and mathematical problems in natural language ( Saxton et al. , 2019 ; Schlag et al. , 2019 ) . Probably the closest work to this paper are the applications of Transformers to directly solve differential equations ( Lample & Charton , 2020 ) and directly predict missing assumptions and types of formal mathematical statements ( Rabe et al. , 2020 ) . We focus on a different problem domain , verification , and demonstrate that Transformers are roughly competitive with classical algorithms in that domain on their dataset . Learning has been applied to mathematics long before the rise of deep learning . Earlier works focused on ranking premises or clauses Cairns ( 2004 ) ; Urban ( 2004 ; 2007 ) ; Urban et al . ( 2008 ) ; Meng & Paulson ( 2009 ) ; Schulz ( 2013 ) ; Kaliszyk & Urban ( 2014 ) . Neural architectures for logical reasoning . ( Paliwal et al. , 2020 ) demonstrate significant improvements in theorem proving through the use of graph neural networks to represent higher-order logic terms . Selsam et al . ( 2019 ) presented NeuroSAT , a graph neural network ( Scarselli et al. , 2008 ; Li et al. , 2017 ; Gilmer et al. , 2017 ; Wu et al. , 2019 ) for solving the propositional satisfiability problem . In contrast , we apply a generic sequence-to-sequence model to predict the solutions to formulas , not only whether there is a solution . This allows us to apply the approach to a wider set of logics ( logics without a CNF ) . A simplified NeuroSAT architecture was trained for unsat-core predictions ( Selsam & Bjørner , 2019 ) . Lederman et al . ( 2020 ) have used graph neural networks on CNF to learn better heuristics for a 2QBF solver . Evans et al . ( 2018 ) study the problem of logical entailment in propositional logic using tree-RNNs . Entailment is a subproblem of satisfiability and ( besides being a classification problem ) could be encoded in the same form as our propositional formulas . The formulas considered in their dataset are much smaller than in this work . Language models applied to programs . Transformers have also been applied to programs for tasks such as summarizing code ( Fernandes et al. , 2018 ) or variable naming and misuse ( Hellendoorn et al. , 2020 ) . Other works focused on recurrent neural networks or graph neural networks for code analysis , e.g . ( Piech et al. , 2015 ; Gupta et al. , 2017 ; Bhatia et al. , 2018 ; Wang et al. , 2018 ; Allamanis et al. , 2017 ) . Another area in the intersection of formal methods and machine learning is the verification of neural networks ( Seshia & Sadigh , 2016 ; Seshia et al. , 2018 ; Singh et al. , 2019 ; Gehr et al. , 2018 ; Huang et al. , 2017 ; Dreossi et al. , 2019 ) . 3 DATASETS . To demonstrate the generalization properties of the Transformer on logical tasks , we generated several datasets in three different fashions . We will describe the underlying logical problems and our data generation in the following . 3.1 TRACE GENERATION FOR LINEAR-TIME TEMPORAL LOGIC . Linear-time temporal logic ( LTL , Pnueli , 1977 ) combines propositional connectives with temporal operators such as the Next operator and the Until operator U . ϕ means that ϕ holds in the next position of a sequence ; ϕ1 U ϕ2 means that ϕ1 holds until ϕ2 holds . For example , the LTL formula ( bU a ) ∧ ( cU ¬a ) states that b has to hold along the trace until a holds and c has to hold until a does not hold anymore . There also exist derived operators . For example , consider the following specification of an arbiter : ( request → grant ) states that , at every point in time ( -operator ) , if there is a request signal , then a grant signal must follow at some future point in time ( -operator ) . The full semantics and an explanation of the operators can be found in Appendix A . We consider infinite sequences , that are finitely represented in the form of a “ lasso ” uvω , where u , called prefix , and v , called period , are finite sequences of propositional formulas . We call such sequences ( symbolic ) traces . For example , the symbolic trace ( a ∧ b ) ω defines the infinite sequence where a and b evaluate to true on every position . Symbolic traces allow us to underspecify propositions when they do not matter . For example , the LTL formula a is satisfied by the symbolic trace : true true ( a ) ω , which allow for any combination of propositions on the first two positions . Our datasets consist of pairs of satisfiable LTL formulas and satisfying symbolic traces generated with tools and automata constructions from the spot framework ( Duret-Lutz et al. , 2016 ) . We use a compact syntax for ultimately periodic symbolic traces : Each position in the trace is separated by the delimiter “ ; ” . True and False are represented by “ 1 ” and “ 0 ” , respectively . The beginning of the period v is signaled by the character “ { ” and analogously its end by “ } ” . For example , the ultimately periodic symbolic trace denoted by a ; a ; a ; { b } , describes all infinite traces where on the first 3 positions a must hold followed by an infinite period on which b must hold on every position . Given a satisfiable LTL formula ϕ , our trace generator constructs a Büchi automatonAϕ that accepts exactly the language defined by the LTL formula , i.e. , L ( Aϕ ) = L ( ϕ ) . From this automaton , we construct an arbitrary accepted symbolic trace , by searching for an accepting run in Aϕ .

Summary: The paper explores the application of modern learning techniques (transformers and tree positional encodings) to the task of producing valid traces for a given LTL specification. A series of experiments are conducted to explore the generalization power of the proposed approach, both in terms of formula size and style of constraint. The work follows a recent trend of the application of deep learning techniques to logic-based settings, and the authors present (to the best of my knowledge) the first attempt of applying deep learning techniques to this particular task.