Learning a Meta-Solver for Syntax-Guided Program Synthesis

A general formulation of program synthesis called syntax-guided synthesis ( SyGuS ) seeks to synthesize a program that follows a given grammar and satisfies a given logical specification . Both the logical specification and the grammar have complex structures and can vary from task to task , posing significant challenges for learning across different tasks . Moreover , supervision is often unavailable for domain-specific synthesis tasks . To address these challenges , we propose a metalearning framework that learns a transferable policy using only weak supervision . Our framework consists of three components : 1 ) an encoder , which embeds both the logical specification and grammar at the same time using a graph neural network ; 2 ) a grammar adaptive policy network which enables learning a transferable policy ; and 3 ) a reinforcement learning algorithm that jointly trains the embedding and adaptive policy with sparse reward . We evaluate the framework on 214 cryptographic circuit synthesis tasks . It solves 141 of them in the out-of-box solver setting , significantly outperforming a similar search-based approach but without learning , which solves only 31 . The result is comparable to two state-of-the-art classical synthesis engines , which solve 129 and 153 respectively . In the metasolver setting , the framework can efficiently adapt to unseen tasks and achieves speedup ranging from 2× up to 100× . 1 INTRODUCTION . Program synthesis concerns automatically generating a program that satisfies desired functional requirements . Promising results have been demonstrated by applying this approach to problems in diverse domains , such as spreadsheet data manipulation for end-users ( Gulwani et al. , 2012 ) , intelligent tutoring for students ( Singh et al. , 2013 ) , and code auto-completion for programmers ( Feng et al. , 2017 ) , among many others . In a common formulation posed by Alur et al . ( 2013 ) called syntax-guided synthesis ( SyGuS ) , the program synthesizer takes as input a logical formula φ and a grammar G , and produces as output a program in G that satisfies φ . In this formulation , φ constitutes a semantic specification that describes the desired functional requirements , and G is a syntactic specification that constrains the space of possible programs . The SyGuS formulation has been targeted by a variety of program synthesizers based on discrete techniques such as constraint solving ( Reynolds et al. , 2015 ) , enumerative search ( Alur et al. , 2017b ) , and stochastic search ( Schkufza et al. , 2013 ) . A key limitation of these synthesizers is that they do not bias their search towards likely programs . This in turn hinders their efficiency and limits the kinds of programs they are able to synthesize . It is well known that likely programs have predictable patterns ( Hindle et al. , 2012 ; Allamanis et al. , 2018a ) . As a result , recent works have leveraged neural networks for program synthesis . However , they are limited in two aspects . First , they do not target general SyGuS tasks ; more specifically : ∗The first two authors contributed equally to this work . • They assume a fixed grammar ( i.e. , syntactic specification G ) across tasks . For example , Si et al . ( 2018 ) learn loop invariants for program verification , but the grammar of loop invariants is fixed across different programs to be verified . • The functional requirements ( i.e. , semantic specification φ ) are omitted , in applications that con- cern identifying semantically similar programs ( Piech et al. , 2015 ; Allamanis et al. , 2017 ; 2018b ; Dai et al. , 2018 ) , or presumed to be input-output examples ( Parisotto et al. , 2016 ; Balog et al. , 2017 ; Devlin et al. , 2017 ; Bunel et al. , 2018 ; Chen et al. , 2018 ; Vijayakumar et al. , 2018 ; Shin et al . ; Sun et al. , 2018 ; Pu et al. , 2018 ) . In contrast , the SyGuS formulation allows the grammar G to vary across tasks , thereby affording flexibility to enforce different syntactic requirements in each task . It also allows to specify functional requirements in a manner more general than input-output examples , by allowing the semantic specification φ to be a logical formula ( e.g. , f ( x ) = 2x instead of f ( 1 ) = 2 ∧ f ( 3 ) = 6 ) . As a result , the general SyGuS setting necessitates the ability to capture common patterns across different specifications and grammars . A second limitation of existing approaches is that they rely on strong supervision on the generated program ( Parisotto et al. , 2016 ; Balog et al. , 2017 ; Bunel et al. , 2018 ) . However , in SyGuS tasks , ground truth programs f are not readily available ; instead , a checker is provided that verifies whether f satisfies φ . In this paper , we propose a framework that is general in that it makes few assumptions on specific grammars or constraints , and has meta-learning capability that can be utilized in solving unseen tasks more efficiently . The key contributions we make are ( 1 ) a joint graph representation of both syntactic and semantic constraints in each task that is learned by a graph neural network model ; ( 2 ) a grammar adaptive policy network that generalizes across different grammars and guides the search for the desired program ; and ( 3 ) a reinforcement learning training method that enables learning transferable representation and policy with weak supervision . We demonstrate our meta-learning framework on a challenging and practical instance of the SyGuS problem that concerns synthesizing cryptographic circuits that are provably free of side-channel attacks ( Eldib et al. , 2016 ) . In our experiments , we first compare the framework in an out-of-box solver setting against a similar search-based approach and two state-of-the-art classical solvers developed in the formal methods community . Then we demonstrate its capability as a meta-solver that can efficiently adapt to unseen tasks , and compare it to the out-of-box version . 2 PROBLEM FORMULATION . The Syntax-Guided Synthesis ( SyGuS ) problem is to synthesize a function f that satisfies two kinds of constraints : • a syntactic constraint specified by a context-free grammar ( CFG ) G , and • a semantic constraint specified by a formula φ built from symbols in a background theory T along with f . One example of the SyGuS problem is cryptographic circuit synthesis ( Eldib et al. , 2016 ) . The goal is to synthesize a side-channel free cryptographic circuit by following the given CFG ( syntactic constraint ) while ensuring that the synthesized circuit is equivalent to the original circuit ( semantic constraint ) . In this example , the grammar is designed to avoid side-channel attacks , whereas the original circuit is created only for functional correctness and thus is vulnerable to such attacks . We henceforth use this problem as an illustrative example but note that our proposed method is not limited to this specific SyGuS problem . We investigate how to efficiently synthesize the function f . Specifically , given a dataset of N tasks D = { ( φi , Gi ) } Ni=1 , we address the following two tasks : • learning an algorithm Aθ : ( φ , G ) 7→ f parameterized by θ that can find the function fi for ( φi , Gi ) ∈ D ; • given a new task set D′ , adapt the above learned algorithmAθ and execute it on new tasks in D′ . This setting poses two difficulties in learning . First , the ground truth target function f is not readily available , making it difficult to formulate as a supervised learning problem . Second , the constraint φ is typically verified using an SAT or SMT solver , and this solver in turns expects the generated f to be complete . This means the weak supervision signal will only be given after the entire program is generated . Thus , it is natural to formulate Aθ as a reinforcement learning algorithm . Since each instance ( φi , Gi ) ∈ D is an independent task with different syntactic and semantic constraints , the key to success is the design of such meta-learner , which we elaborate in Sec 3 . 3 META-SOLVER MODEL . This section presents our meta-solver model for solving the two problems formulated in Sec 2 . We first introduce formal notation in Sec 3.1 . To enable the transfer of knowledge across tasks with different syntactic and semantic constraints , we propose a representation framework in Sec 3.2 to jointly encode the two kinds of constraints . The representation needs to be general enough to encode constraints with different specifications . Lastly , we introduce the Grammar Adaptive Policy Network in Sec 3.3 that executes a program generation policy while automatically adapting to different grammars encoded in each task specification . 3.1 FORMAL DEFINITIONS . We formally define key concepts in the SyGuS problem formulation as follows . semantic spec φ : The spec itself is a program written using some grammar . In our case , the grammar used in spec φ is different from the grammar G that specifies the syntax of the output program . However , in many practical cases the tokens ( i.e. , the dictionary of terminal symbols ) may be shared across the input spec and the output program . CFG G : A context free grammar ( CFG ) is defined as G = 〈V , Σ , R , s〉 . Here V denotes the nonterminal tokens , while Σ represents the terminal tokens . s is a special token that denotes the start of the language , and the language is generated according to the production rules defined in R. For a given non-terminal , the associated production rules can be written as α → β1|β2 . . . |βnα , where nα is the branching factor for non-terminal α ∈ V , and βi = u1u2 . . . u|βi| ∈ ( V ⋃ Σ ) ∗ . Each production rule α → βi ∈ R represents a way of expanding the grammar tree , by attaching nodes u1 , u2 , . . . , u|βi| to node α . The expansion is repeated until all the leaf nodes are terminals . Output function f : The output is a program in the language generated by G. A valid output f must satisfy both the syntactic constraints specified by G and the semantic constraints specified by φ . 3.2 TASK REPRESENTATION . Different from traditional neural program synthesis tasks , where the program grammar and vocabulary is fixed , each individual task in our setting has its own form of grammar and semantic specification . Thus in the program generation phase ( which we will elucidate in Sec 3.3 ) , one can not assume a fixed CFG and use a tree decoder like in Kusner et al . ( 2017 ) and Bunel et al . ( 2018 ) . To enable such generalization across different grammars , the information about the CFG for each task needs to be captured in the task representation . Since the semantic spec program φ and the CFG G have rich structural information , it is natural to use graphs for their representation . Representing the programs using graphs has been successfully used in many programming language domains . In our work , we further extend the approach by Allamanis et al . ( 2018b ) with respect to the following aspects : • Instead of only representing the semantic spec program φ as a graph , we propose to jointly represent it along with the grammar G. • To allow information exchange between the two graphs , we leverage the idea of Static Single Assignment ( SSA ) form in compiler design . That is , the same variable ( token ) that may be assigned ( defined ) at many different places should be viewed differently , but on the other hand , these variations correspond to the same original thing . Specifically , we introduce global nodes for shared tokens and global links connecting these globally shared nodes and local nodes that ( re ) define corresponding tokens . The overall representation framework is described in Figure 1 . To construct the graph , we first build the abstract syntax tree ( AST ) for the semantic spec program φ , according to its own grammar ( typically different from the output grammar G ) . To represent the grammar G , we associate each symbol in V ⋃ Σ with a node representation . Furthermore , for a non-terminal α and its corresponding production rules α→ β1|β2 . . . |βnα , we create additional nodes αi for each substitute βi . The purpose is to enable grammar adaptive program generation , which we elaborate in Sec 3.3 . As a simplification , we merge all nodes αi representing βi that is a single terminal token into one node . Finally , the global nodes for shared tokens in Σ are created to link together the shared variable and operator nodes . This enables information exchange between the syntactic and semantics specifications . To encode the joint graph G ( φ , G ) , we use graph neural networks to get the vector representation for each node in the graph . Specifically , for each node v ∈ G , we use the following parameterization for one step of message passing style update : ht+1v = Aggregate ( { F ( htu , eu , v ) } u∈N ( v ) ) ( 1 ) Lastly , { hTv } v∈G , hTv ∈ Rd are the set of node embeddings . Here N ( v ) is the set of neighbor nodes of v , and eu , v denotes the type of edge that links the node u and v. We parameterize F in a way similar to GGNN ( Li et al. , 2015 ) , i.e. , F ( ht , e ) = σ ( W e > t h t ) where we use different matrices W ∈ Rd×d for different edge types and different propagation steps t. We sum over all the node embeddings to get the global graph embedding h ( G ) . In addition to the node embeddings and global graph embedding , we also obtain the embedding matrix for each non-terminal node . Specifically , given node α , we will have the embedding matrix Hα ∈ Rnα×d , where the ith row ofH ( i ) α is the embedding of node αi that corresponds to substitution βi . This enables the grammar adaptive tree expansion in Sec 3.3 .

Summary: This paper presents a reinforcement learning based approach to learn a search strategy to search for programs in the generic syntax-guided synthesis (SyGuS) formulation. Unlike previous neural program synthesis approaches, where the DSL grammar is fixed or the specification is in the form of input-output examples only, the SyGuS formulation considers different grammars for different synthesis problems and the specification format is also more general. The main idea of the approach is to first learn a joint representation of the specification and grammar using a graph neural network model, and then train a policy using reinforcement learning to guide the search with a grammar adaptive policy network that is conditioned on the joint representation. Since the specifications considered here are richer logical expressions, it uses a SAT solver for checking the validity of the proposed solution and to also obtain counterexamples for future rewards. The technique is evaluated on 210 SyGuS benchmarks coming from the cryptographic circuit synthesis domain, and shows significant improvements in terms of number of instances solved compared to CVC4 and ESymbolic baseline search techniques from the formal methods community. Moreover, the learnt policy is also showed to generalize beyond the benchmarks on which it is trained and the meta-solver performs reasonably well compared to the per-task out-of-box solver.