Learning a Meta-Solver for Syntax-Guided Program Synthesis

A general formulation of program synthesis called syntax-guided synthesis ( SyGuS ) seeks to synthesize a program that follows a given grammar and satisfies a given logical specification . Both the logical specification and the grammar have complex structures and can vary from task to task , posing significant challenges for learning across different tasks . Moreover , supervision is often unavailable for domain-specific synthesis tasks . To address these challenges , we propose a metalearning framework that learns a transferable policy using only weak supervision . Our framework consists of three components : 1 ) an encoder , which embeds both the logical specification and grammar at the same time using a graph neural network ; 2 ) a grammar adaptive policy network which enables learning a transferable policy ; and 3 ) a reinforcement learning algorithm that jointly trains the embedding and adaptive policy with sparse reward . We evaluate the framework on 214 cryptographic circuit synthesis tasks . It solves 141 of them in the out-of-box solver setting , significantly outperforming a similar search-based approach but without learning , which solves only 31 . The result is comparable to two state-of-the-art classical synthesis engines , which solve 129 and 153 respectively . In the metasolver setting , the framework can efficiently adapt to unseen tasks and achieves speedup ranging from 2× up to 100× . 1 INTRODUCTION . Program synthesis concerns automatically generating a program that satisfies desired functional requirements . Promising results have been demonstrated by applying this approach to problems in diverse domains , such as spreadsheet data manipulation for end-users ( Gulwani et al. , 2012 ) , intelligent tutoring for students ( Singh et al. , 2013 ) , and code auto-completion for programmers ( Feng et al. , 2017 ) , among many others . In a common formulation posed by Alur et al . ( 2013 ) called syntax-guided synthesis ( SyGuS ) , the program synthesizer takes as input a logical formula φ and a grammar G , and produces as output a program in G that satisfies φ . In this formulation , φ constitutes a semantic specification that describes the desired functional requirements , and G is a syntactic specification that constrains the space of possible programs . The SyGuS formulation has been targeted by a variety of program synthesizers based on discrete techniques such as constraint solving ( Reynolds et al. , 2015 ) , enumerative search ( Alur et al. , 2017b ) , and stochastic search ( Schkufza et al. , 2013 ) . A key limitation of these synthesizers is that they do not bias their search towards likely programs . This in turn hinders their efficiency and limits the kinds of programs they are able to synthesize . It is well known that likely programs have predictable patterns ( Hindle et al. , 2012 ; Allamanis et al. , 2018a ) . As a result , recent works have leveraged neural networks for program synthesis . However , they are limited in two aspects . First , they do not target general SyGuS tasks ; more specifically : ∗The first two authors contributed equally to this work . • They assume a fixed grammar ( i.e. , syntactic specification G ) across tasks . For example , Si et al . ( 2018 ) learn loop invariants for program verification , but the grammar of loop invariants is fixed across different programs to be verified . • The functional requirements ( i.e. , semantic specification φ ) are omitted , in applications that con- cern identifying semantically similar programs ( Piech et al. , 2015 ; Allamanis et al. , 2017 ; 2018b ; Dai et al. , 2018 ) , or presumed to be input-output examples ( Parisotto et al. , 2016 ; Balog et al. , 2017 ; Devlin et al. , 2017 ; Bunel et al. , 2018 ; Chen et al. , 2018 ; Vijayakumar et al. , 2018 ; Shin et al . ; Sun et al. , 2018 ; Pu et al. , 2018 ) . In contrast , the SyGuS formulation allows the grammar G to vary across tasks , thereby affording flexibility to enforce different syntactic requirements in each task . It also allows to specify functional requirements in a manner more general than input-output examples , by allowing the semantic specification φ to be a logical formula ( e.g. , f ( x ) = 2x instead of f ( 1 ) = 2 ∧ f ( 3 ) = 6 ) . As a result , the general SyGuS setting necessitates the ability to capture common patterns across different specifications and grammars . A second limitation of existing approaches is that they rely on strong supervision on the generated program ( Parisotto et al. , 2016 ; Balog et al. , 2017 ; Bunel et al. , 2018 ) . However , in SyGuS tasks , ground truth programs f are not readily available ; instead , a checker is provided that verifies whether f satisfies φ . In this paper , we propose a framework that is general in that it makes few assumptions on specific grammars or constraints , and has meta-learning capability that can be utilized in solving unseen tasks more efficiently . The key contributions we make are ( 1 ) a joint graph representation of both syntactic and semantic constraints in each task that is learned by a graph neural network model ; ( 2 ) a grammar adaptive policy network that generalizes across different grammars and guides the search for the desired program ; and ( 3 ) a reinforcement learning training method that enables learning transferable representation and policy with weak supervision . We demonstrate our meta-learning framework on a challenging and practical instance of the SyGuS problem that concerns synthesizing cryptographic circuits that are provably free of side-channel attacks ( Eldib et al. , 2016 ) . In our experiments , we first compare the framework in an out-of-box solver setting against a similar search-based approach and two state-of-the-art classical solvers developed in the formal methods community . Then we demonstrate its capability as a meta-solver that can efficiently adapt to unseen tasks , and compare it to the out-of-box version . 2 PROBLEM FORMULATION . The Syntax-Guided Synthesis ( SyGuS ) problem is to synthesize a function f that satisfies two kinds of constraints : • a syntactic constraint specified by a context-free grammar ( CFG ) G , and • a semantic constraint specified by a formula φ built from symbols in a background theory T along with f . One example of the SyGuS problem is cryptographic circuit synthesis ( Eldib et al. , 2016 ) . The goal is to synthesize a side-channel free cryptographic circuit by following the given CFG ( syntactic constraint ) while ensuring that the synthesized circuit is equivalent to the original circuit ( semantic constraint ) . In this example , the grammar is designed to avoid side-channel attacks , whereas the original circuit is created only for functional correctness and thus is vulnerable to such attacks . We henceforth use this problem as an illustrative example but note that our proposed method is not limited to this specific SyGuS problem . We investigate how to efficiently synthesize the function f . Specifically , given a dataset of N tasks D = { ( φi , Gi ) } Ni=1 , we address the following two tasks : • learning an algorithm Aθ : ( φ , G ) 7→ f parameterized by θ that can find the function fi for ( φi , Gi ) ∈ D ; • given a new task set D′ , adapt the above learned algorithmAθ and execute it on new tasks in D′ . This setting poses two difficulties in learning . First , the ground truth target function f is not readily available , making it difficult to formulate as a supervised learning problem . Second , the constraint φ is typically verified using an SAT or SMT solver , and this solver in turns expects the generated f to be complete . This means the weak supervision signal will only be given after the entire program is generated . Thus , it is natural to formulate Aθ as a reinforcement learning algorithm . Since each instance ( φi , Gi ) ∈ D is an independent task with different syntactic and semantic constraints , the key to success is the design of such meta-learner , which we elaborate in Sec 3 . 3 META-SOLVER MODEL . This section presents our meta-solver model for solving the two problems formulated in Sec 2 . We first introduce formal notation in Sec 3.1 . To enable the transfer of knowledge across tasks with different syntactic and semantic constraints , we propose a representation framework in Sec 3.2 to jointly encode the two kinds of constraints . The representation needs to be general enough to encode constraints with different specifications . Lastly , we introduce the Grammar Adaptive Policy Network in Sec 3.3 that executes a program generation policy while automatically adapting to different grammars encoded in each task specification . 3.1 FORMAL DEFINITIONS . We formally define key concepts in the SyGuS problem formulation as follows . semantic spec φ : The spec itself is a program written using some grammar . In our case , the grammar used in spec φ is different from the grammar G that specifies the syntax of the output program . However , in many practical cases the tokens ( i.e. , the dictionary of terminal symbols ) may be shared across the input spec and the output program . CFG G : A context free grammar ( CFG ) is defined as G = 〈V , Σ , R , s〉 . Here V denotes the nonterminal tokens , while Σ represents the terminal tokens . s is a special token that denotes the start of the language , and the language is generated according to the production rules defined in R. For a given non-terminal , the associated production rules can be written as α → β1|β2 . . . |βnα , where nα is the branching factor for non-terminal α ∈ V , and βi = u1u2 . . . u|βi| ∈ ( V ⋃ Σ ) ∗ . Each production rule α → βi ∈ R represents a way of expanding the grammar tree , by attaching nodes u1 , u2 , . . . , u|βi| to node α . The expansion is repeated until all the leaf nodes are terminals . Output function f : The output is a program in the language generated by G. A valid output f must satisfy both the syntactic constraints specified by G and the semantic constraints specified by φ . 3.2 TASK REPRESENTATION . Different from traditional neural program synthesis tasks , where the program grammar and vocabulary is fixed , each individual task in our setting has its own form of grammar and semantic specification . Thus in the program generation phase ( which we will elucidate in Sec 3.3 ) , one can not assume a fixed CFG and use a tree decoder like in Kusner et al . ( 2017 ) and Bunel et al . ( 2018 ) . To enable such generalization across different grammars , the information about the CFG for each task needs to be captured in the task representation . Since the semantic spec program φ and the CFG G have rich structural information , it is natural to use graphs for their representation . Representing the programs using graphs has been successfully used in many programming language domains . In our work , we further extend the approach by Allamanis et al . ( 2018b ) with respect to the following aspects : • Instead of only representing the semantic spec program φ as a graph , we propose to jointly represent it along with the grammar G. • To allow information exchange between the two graphs , we leverage the idea of Static Single Assignment ( SSA ) form in compiler design . That is , the same variable ( token ) that may be assigned ( defined ) at many different places should be viewed differently , but on the other hand , these variations correspond to the same original thing . Specifically , we introduce global nodes for shared tokens and global links connecting these globally shared nodes and local nodes that ( re ) define corresponding tokens . The overall representation framework is described in Figure 1 . To construct the graph , we first build the abstract syntax tree ( AST ) for the semantic spec program φ , according to its own grammar ( typically different from the output grammar G ) . To represent the grammar G , we associate each symbol in V ⋃ Σ with a node representation . Furthermore , for a non-terminal α and its corresponding production rules α→ β1|β2 . . . |βnα , we create additional nodes αi for each substitute βi . The purpose is to enable grammar adaptive program generation , which we elaborate in Sec 3.3 . As a simplification , we merge all nodes αi representing βi that is a single terminal token into one node . Finally , the global nodes for shared tokens in Σ are created to link together the shared variable and operator nodes . This enables information exchange between the syntactic and semantics specifications . To encode the joint graph G ( φ , G ) , we use graph neural networks to get the vector representation for each node in the graph . Specifically , for each node v ∈ G , we use the following parameterization for one step of message passing style update : ht+1v = Aggregate ( { F ( htu , eu , v ) } u∈N ( v ) ) ( 1 ) Lastly , { hTv } v∈G , hTv ∈ Rd are the set of node embeddings . Here N ( v ) is the set of neighbor nodes of v , and eu , v denotes the type of edge that links the node u and v. We parameterize F in a way similar to GGNN ( Li et al. , 2015 ) , i.e. , F ( ht , e ) = σ ( W e > t h t ) where we use different matrices W ∈ Rd×d for different edge types and different propagation steps t. We sum over all the node embeddings to get the global graph embedding h ( G ) . In addition to the node embeddings and global graph embedding , we also obtain the embedding matrix for each non-terminal node . Specifically , given node α , we will have the embedding matrix Hα ∈ Rnα×d , where the ith row ofH ( i ) α is the embedding of node αi that corresponds to substitution βi . This enables the grammar adaptive tree expansion in Sec 3.3 .

Summary: This paper presents a (meta-)solver for particular program synthesis problems, where the model has access to a (logic) specification of the program to be synthesized, and a grammar that can change from one task instance to another. The presented model is an RL-based model that jointly trains 1) the joint graph-based embedding of the specification and the grammar, and 2) a policy able to operate on different (from instance to instance) grammars. Interestingly, not only can the model operate as a stand-alone solver, but it can be run as a meta-solver - trained on a subset of tasks, and applied (with tuning) on a new task. Experiments show that the model outperforms two baselines (one being a (near-to-)SOTA model) in the stand-alone setting and that the model successfully transfers knowledge (considers fewer candidates) in the meta-solving mode.