Original: L EARNING

Original: A

Original:  M ETA -S OLVER FOR S YNTAX -G UIDED P ROGRAM S YNTHESIS
Revised: L EARNING A M ETA -S OLVER FOR S YNTAX -G UIDED P ROGRAM S YNTHESIS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: A general formulation of program synthesis called syntax-guided synthesis (SyGuS) seeks to synthesize a program that follows a given grammar and satisﬁes a given logical speciﬁcation.
Revised: A general formulation of program synthesis called syntax-guided synthesis (Sy- GuS ) seeks to synthesize a program that follows a given grammar and satisﬁes a given logical speciﬁcation.

Original: Both the logical speciﬁcation and the grammar have complex structures and can vary from task to task, posing signiﬁcant challenges for learning across different tasks.
Revised: Both the logical speciﬁcation and the grammar have complex structures and can vary from task to task, posing signiﬁcant challenges for learning across different tasks.

Original: Furthermore, supervision is often unavailable for domain-speciﬁc synthesis tasks.
Revised: Moreover, supervision is often unavailable for domain-speciﬁc synthesis tasks.

Original: To address these challenges, we propose a meta-learning framework that learns a transferable policy from only weak supervision.
Revised: To address these challenges, we propose a metalearning framework that learns a transferable policy using only weak supervision

Original: Our framework consists of three components: 1) an encoder, which embeds both the logical speciﬁcation and grammar at the same time using a graph neural network; 2) a grammar adaptive policy network which enables learning a transferable policy; and 3) a reinforcement learning algorithm that jointly trains the speciﬁcation and grammar embedding and adaptive policy with sparse reward.
Revised: Our framework consists of three components: 1) an encoder which embeds both the logical speciﬁcation and grammar at the same time using a graph neural network; 2) a grammar adaptive policy network which enables learning a transferable policy; and 3) a reinforcement learning algorithm that jointly trains the speciﬁcation and grammar embedding and adaptive policy with sparse reward.

Original: We evaluate the framework on 214 cryptographic circuit synthesis tasks.
Revised: We evaluate the framework on 214 cryptographic circuit synthesis tasks.

Original: It solves 141 of them in the out-of-box solver setting, signiﬁcantly outperforming a similar search-based approach but without learning, which solves only 31.
Revised: It solves 141 of them in the out-of-box solver setting, signiﬁcantly outperforming a similar search-based approach but without learning which solves only 31.

Original: The result is comparable to two state-of-the-art classical synthesis engines, which solveand 153 respectively.
Revised: The result is comparable to two state-of-the-art classical synthesis engines, which solve 129 and 153 respectively.

Original: In the meta-solver setting, the framework can efﬁciently adapt to unseen tasks and achieves speedup ranging from 2 × up to100 × .
Revised: In the meta-solver setting, the framework can efﬁciently adapt to unseen tasks and achieves speedup ranging from 2 × up to × .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Program synthesis concerns automatically generating a program that satisﬁes desired functional requirements.
Revised: Program synthesis concerns automatically generating a program that satisﬁes desired functional requirements.

Original: Promising results have been demonstrated by applying this approach to problems in diverse domains, such as spreadsheet data manipulation for end-users (Gulwani et al., 2012), intelligent tutoring for students (Singh et al., 2013), and code auto-completion for programmers (Feng et al., 2017), among many others.
Revised: Promising results have been demonstrated by applying this approach to problems in diverse domains, such as spreadsheet data manipulation for end-users (Gulwani et al., 2012), intelligent tutoring for students (Singh et al., 2013), and code auto-completion for programmers (Feng et al., 2017), among many others.

Original: In a common formulation posed by Alur et al.
Revised: In a common formulation posed by Alur et al.

Original: called syntax-guided synthesis (SyGuS), the program synthesizer takes as input a logical formula φ and a grammar G , and produces as output a program in G that satisﬁes φ .
Revised: called syntax-guided synthesis (SyGuS), the program synthesizer takes as input a logical formula φ and a grammar G , and produces as output a program in G that satisﬁes φ .

Original: In this formulation, φ constitutes a semantic speciﬁcation that describes the desired functional requirements, and G is a syntactic speciﬁcation that constrains the space of possible programs.
Revised: In this formulation, φ constitutes a semantic speciﬁcation that describes the desired functional requirements, and G is a syntactic speciﬁcation that constrains the space of possible programs.

Original: The SyGuS formulation has been targeted by a variety of program synthesizers based on discrete techniques such as constraint solving (Reynolds et al., 2015), enumerative search (Alur et al., 2017b), and stochastic search (Schkufza et al., 2013).
Revised: The SyGuS formulation has been targeted by a variety of program synthesizers based on discrete techniques such as constraint solving (Reynolds et al., 2015), enumerative search (Alur et al., 2017b), and stochastic search (Schkufza et al., 2013).

Original: A key limitation of these synthesizers is that they do not bias their search towards likely programs.
Revised: A key limitation of these synthesizers is that they do not bias their search towards likely programs.

Original: This in turn hinders their efﬁciency and limits the kinds of programs they are able to synthesize.
Revised: This in turn hinders their efﬁciency and limits the kinds of programs they are able to synthesize.

Original: It is well known that likely programs have predictable patterns (Hindle et al., 2012; Allamanis et al., 2018a).
Revised: It is well known that likely programs have predictable patterns (Hindle et al., 2012; Allamanis et al., 2018a).

Original: As a result, recent works have leveraged neural networks for program synthesis.
Revised: As a result, recent works have leveraged neural networks for program synthesis.

Original: However, they are limited in two aspects.
Revised: However, they are limited in two aspects.

Original: First, they do not target general SyGuS tasks; more speciﬁcally:
Revised: First, they do not target general SyGuS tasks; more speciﬁcally:

Original: • They assume a ﬁxed grammar (i e., syntactic speciﬁcation G ) across tasks.
Revised: • They assume a ﬁxed grammar (i.e., syntactic speciﬁcation G ) across tasks.

Original: For example, Si et al.
Revised: For example, Si et al.

Original: (2018) learn loop invariants for program veriﬁcation, but the grammar of loop invariants is ﬁxed across different programs to be veriﬁed.
Revised: (2018) learn loop invariants for program veriﬁcation, but the grammar of loop invariants is ﬁxed across different programs to be veriﬁed.

Original: • The functional requirements (i.e., semantic speciﬁcation φ ) are omitted, in applications that concern identifying semantically similar programs (Piech et al., 2015; Allamanis et al., 2017; 2018b; Dai et al., 2018), or presumed to be input-output examples (Parisotto et al., 2016;
Revised: • The functional requirements (i e., semantic speciﬁcation φ ) are omitted, in applications that concern identifying semantically similar programs (Piech et al., 2015; Allamanis et al., 2017; 2018b; Dai et al., 2018), or presumed to be input-output examples (Parisotto et al., 2016;

Original: Balog et al., 2017; Devlin et al., 2017; Bunel et al., 2018; Chen et al., 2018; Vijayakumar et al., 2018; Shin et al.
Revised: Balog et al., 2017; Devlin et al., 2017; Bunel et al., 2018; Chen et al., 2018; Vijayakumar et al., 2018; Shin et al.

Original: ; Sun et al., 2018; Pu et al., 2018).
Revised: ; Sun et al., 2018; Pu et al., 2018).

Original: In contrast, the SyGuS formulation allows the grammar G to vary across tasks, thereby affording ﬂexibility to enforce different syntactic requirements in each task.
Revised: In contrast, the SyGuS formulation allows the grammar G to vary across tasks, thereby affording ﬂexibility to enforce different syntactic requirements in each task.

Original: It also allows to specify functional requirements in a manner more general than input-output examples, by allowing the semantic speciﬁcation φ to be a logical formula (e.g., f ( x ) = 2 x instead of f (1)
Revised: It also allows to specify functional requirements in a manner more general than input-output examples, by allowing the semantic speciﬁcation φ to be a logical formula (e.g., f ( x ) = 2 x instead of f (1)

Original: = 

Revised: =

Original: ∧ f (3) = 6 ).
Revised: ∧ f (3) = 6 ).

Original: As a result, the general SyGuS setting necessitates the ability to capture common patterns across different speciﬁcations and grammars.
Revised: As a result, the general SyGuS setting necessitates the ability to capture common patterns across different speciﬁcations and grammars.

Original: A second limitation of existing approaches is that they rely on strong supervision on the generated program (Parisotto et al., 2016;
Revised: A second limitation of existing approaches is that they rely on strong supervision on the generated program (Parisotto et al., 2016;

Original: Balog et al., 2017; Bunel et al., 2018).
Revised: Balog et al., 2017; Bunel et al., 2018).

Original: However, in SyGuS tasks, ground truth programs f are not readily available; instead, a checker is provided that veriﬁes whether f satisﬁes φ .
Revised: However, in SyGuS tasks, ground truth programs f are not readily available; instead, a checker is provided that veriﬁes whether f satisﬁes φ .

Original: In this paper, we propose a framework that is general in that it makes few assumptions on speciﬁc grammars or constraints, and has meta-learning capability that can be utilized in solving unseen tasks more efﬁciently.
Revised: In this paper, we propose a framework that is general in that it makes few assumptions on speciﬁc grammars or constraints, and has meta-learning capability that can be utilized in solving unseen tasks more efﬁciently.

Original: The key contributions we make are (1) a joint graph representation of both syntactic and semantic constraints in each task that is learned by a graph neural network model; (2) a grammar adaptive policy network that generalizes across different grammars and guides the search for the desired program; and (3) a reinforcement learning training method that enables learning transferable representation and policy with weak supervision.
Revised: The key contributions we make are (1) a joint graph representation of both syntactic and semantic constraints in each task that is learned by a graph neural network model; (2) a grammar adaptive policy network that generalizes across different grammars and guides the search for the desired program; and (3) a reinforcement learning training method that enables learning transferable representation and policy with weak supervision.

Original: We demonstrate our meta-learning framework on a challenging and practical instance of the SyGuS problem that concerns synthesizing cryptographic circuits that are provably free of side-channel attacks (Eldib et al., 2016).
Revised: We demonstrate our meta-learning framework on a challenging and practical instance of the SyGuS problem that concerns synthesizing cryptographic circuits that are provably free of side-channel attacks (Eldib et al., 2016).

Original: In our experiments, we ﬁrst compare the framework in an out-of-box solver setting against a similar search-based approach and two state-of-the-art classical solvers developed in the formal methods community.
Revised: In our experiments, we ﬁrst compare the framework in an out-of-box solver setting against a similar search-based approach and two state-of-the-art classical solvers developed in the formal methods community.

Original: Then we demonstrate its capability as a meta-solver that can efﬁciently adapt to unseen tasks, and compare it to the out-of-box version.
Revised: Then we demonstrate its capability as a meta-solver that can efﬁciently adapt to unseen tasks, and compare it to the out-of-box version.

Original:  P ROBLEM F ORMULATION
Revised:  P ROBLEM F ORMULATION

Original: The Syntax-Guided Synthesis (SyGuS) problem is to synthesize a function f that satisﬁes two kinds of constraints:
Revised: The Syntax-Guided Synthesis (SyGuS) problem is to synthesize a function f that satisﬁes two kinds of constraints:

Original: • a syntactic constraint speciﬁed by a context-free grammar (CFG) G , and • a semantic constraint speciﬁed by a formula φ built from symbols in a background theory T along with f .
Revised: • a syntactic constraint speciﬁed by a context-free grammar (CFG) G , and • a semantic constraint speciﬁed by a formula φ built from symbols in a background theory T along with f .

Original: One example of the SyGuS problem is cryptographic circuit synthesis (Eldib et al., 2016).
Revised: One example of the SyGuS problem is cryptographic circuit synthesis (Eldib et al., 2016).

Original: The goal is to synthesize a side-channel free cryptographic circuit by following the given CFG (syntactic constraint) while ensuring that the synthesized circuit is equivalent to the original circuit (semantic constraint).
Revised: The goal is to synthesize a side-channel free cryptographic circuit by following the given CFG (syntactic constraint) while ensuring that the synthesized circuit is equivalent to the original circuit (semantic constraint).

Original: In this example, the grammar is designed to avoid side-channel attacks, whereas the original circuit is created only for functional correctness and thus is vulnerable to such attacks.
Revised: In this example, the grammar is designed to avoid side-channel attacks, whereas the original circuit is created only for functional correctness and thus is vulnerable to such attacks.

Original: We henceforth use this problem as an illustrative example but note that our proposed method is not limited to this speciﬁc SyGuS problem.
Revised: We henceforth use this problem as an illustrative example but note that our proposed method is not limited to this speciﬁc SyGuS problem.

Original: We investigate how to efﬁciently synthesize the function f .
Revised: We investigate how to efﬁciently synthesize the function f .

Original: Speciﬁcally, given a dataset of N tasks
Revised: Speciﬁcally, given a dataset of N tasks

Original: D = { ( φ i , G i ) } Ni =1 , we address the following two tasks:
Revised: D = { ( φ i , G i ) } Ni =1 , we address the following two tasks:

Original: • learning an algorithm A θ : ( φ, G ) (cid:55)→ f parameterized by θ that can ﬁnd the function f
Revised: • learning an algorithm A θ : ( φ, G ) (cid:55)→ f parameterized by θ that can ﬁnd the function f

Original: i for ( φ i , G i ) ∈ D ; • given a new task set D (cid:48) , adapt the above learned algorithm
Revised: i for ( φ i , G i ) ∈ D ; • given a new task set D (cid:48) , adapt the above learned algorithm

Original: A θ and execute it on new tasks in D (cid:48) .
Revised: A θ and execute it on new tasks in D (cid:48) .

Original: This setting poses two difﬁculties in learning.
Revised: This setting poses two difﬁculties in learning.

Original: First, the ground truth target function f is not readily available, making it difﬁcult to formulate as a supervised learning problem.
Revised: First, the ground truth target function f is not readily available, making it difﬁcult to formulate as a supervised learning problem.

Original: Second, the constraintφ is typically veriﬁed using an SAT or SMT solver, and this solver in turns expects the generated f to be complete. This means the weak supervision signal will only be given after the entire program is generated. Thus, it is natural to formulate A θ as a reinforcement learning algorithm. Since each instance ( φ i , G i ) ∈ D is an independent task with different syntactic and semantic constraints, the key to success is the design of such meta-learner, which we elaborate in Sec 3.
Revised: Second, the constraint

Original: Global node Non-terminals Terminals
Revised: Global node Non-terminals Terminals

Original: Graph Representation !
Revised: Graph Representation !

Original: Typed AST AST edge Global link
Revised: Typed AST AST edge Global link

Original: Logical Spec #
Revised: Logical Spec #

Original: X OR Y AND Z
Revised: X OR Y AND Z

Original: Grammar % s -> d 1 OR d 1 | d 1 AND d 1 d 1 -> X | Y | d 2 OR dd 2 -> ......
Revised: Grammar % s -> d 1 OR d 1 | d 1 AND d 1 d 1 -> X | Y | d 2 OR dd 2 -> ......

Original: X X d<spec>
Revised: X X d<spec>

Original: OR
Revised: OR

Original: AND
Revised: AND

Original: Y
Revised: Y

Original: Z
Revised: Z

Original: X
Revised: X

Original: S
Revised: S

Original: S_OR
Revised: S_OR

Original: S_AND d
Revised: S_AND d

Original: N T _ D N T
Revised: N T _ D N T

Original: _ D
Revised: _ D

Original: A N D _ D A N D
Revised: A N D _ D A N D

Original: D O R
Revised: D O R

Original: D O R
Revised: D O R

Original: _ D
Revised: _ D

Original: Y X d 1 T N T
Revised: Y X d 1 T N T

Original: _ D d 1 _or d
Revised: _ D d 1 _or d

Original: N T
Revised: N T

Original: _ D
Revised: _ D

Original: O R _
Revised: O R _

Original: D O R
Revised: D O R

Original: _ D T _ D T
Revised: _ D T _ D T

Original: _ D
Revised: _ D

Original: X
Revised: X

Original: Y
Revised: Y

Original: Z
Revised: Z

Original: AND
Revised: AND

Original: OR
Revised: OR

Original: X_D <spec> OR AND
Revised: X_D <spec> OR AND

Original: Y Z X
Revised: Y Z X

Original: S N I _ D N I _ D
Revised: S N I _ D N I _ D

Original: Y X d 2 O R _
Revised: Y X d 2 O R _

Original: D O R
Revised: D O R

Original: _ D T
Revised: _ D T

Original: _ D T
Revised: _ D T

Original: _ D X Y Z
Revised: _ D X Y Z

Original: AND OR <spec> OR
Revised: AND OR <spec> OR

Original: Y Z X
Revised: Y Z X

Original: S S_OR d 1 N I
Revised: S S_OR d 1 N I

Original: D N I _ D A N D
Revised: D N I _ D A N D

Original: _ D A N D
Revised: _ D A N D

Original: D O R D O R
Revised: D O R D O R

Original: _ D
Revised: _ D

Original: Y X d 1 T N I _ D d 1 _or d 2 N I _
Revised: Y X d 1 T N I _ D d 1 _or d 2 N I _

Original: D O R
Revised: D O R

Original: D O R
Revised: D O R

Original: _ D T
Revised: _ D T

Original: _ D T
Revised: _ D T

Original: _ D X Y Z
Revised: _ D X Y Z

Original: OR <spec> OR AND
Revised: OR <spec> OR AND

Original: Y Z X
Revised: Y Z X

Original: S S_OR
Revised: S S_OR

Original: S_AND N I _ D N I _ D A N D
Revised: S_AND N I _ D N I _ D A N D

Original: D A N D
Revised: D A N D

Original: D O R
Revised: D O R

Original: D O R
Revised: D O R

Original: _ D d 1 _or d 2 N I _
Revised: _ D d 1 _or d 2 N I _

Original: D O R
Revised: D O R

Original: D O R
Revised: D O R

Original: _ D X Y Z
Revised: _ D X Y Z

Original: AND OR ......
Revised: AND OR ......

Original: S_OR
Revised: S_OR

Original: S_AND d 1 A N D
Revised: S_AND d 1 A N D

Original: D A N D
Revised: D A N D

Original: D O R
Revised: D O R

Original: D O R
Revised: D O R

Original: _ D d 1 T N I _ D d 1 _or N I _ D
Revised: _ D d 1 T N I _ D d 1 _or N I _ D

Original: S_AND AND AND d
Revised: S_AND AND AND d

Original: Y X d 1 T N I _ D T
Revised: Y X d 1 T N I _ D T

Original: _ D T
Revised: _ D T

Original: _ D
Revised: _ D

Original: Repeat ' times • Global graph embedding: • Node embeddings:
Revised: Repeat ' times • Global graph embedding: • Node embeddings:

Original: Output ...... s dz • 
Revised: Output ...... s dz • Embed

Original: [Figure]
Revised: [Figure]

Revised: φ is typically veriﬁed using an SAT or SMT solver, and this solver in turns expects the generated f to be complete.

Original: [Figure]
Revised: This means the weak supervision signal will only be given after the entire program is generated. Thus, it is natural to formulate A θ as a reinforcement learning algorithm. Since each instance ( φ i , G i ) ∈ D is an independent task with different syntactic and semantic constraints, the key to success is the design of such meta-learner, which we elaborate in Sec 3.  M ETA - SOLVER M ODEL

Original: This section presents our meta-solver model for solving the two problems formulated in Sec 2.
Revised: This section presents our meta-solver model for solving the two problems formulated in Sec 2.

Original: We ﬁrst introduce formal notation in Sec 3.1.
Revised: We ﬁrst introduce formal notation in Sec 3.1.

Original: To enable the transfer of knowledge across tasks with different syntactic and semantic constraints, we propose a representation framework in Sec 3.2 to jointly encode the two kinds of constraints.
Revised: To enable the transfer of knowledge across tasks with different syntactic and semantic constraints, we propose a representation framework in Sec 3.2 to jointly encode the two kinds of constraints.

Original: The representation needs to be general enough to encode constraints with different speciﬁcations.
Revised: The representation needs to be general enough to encode constraints with different speciﬁcations.

Original: Lastly, we introduce the Grammar Adaptive Policy
Revised: Lastly, we introduce the Grammar Adaptive Policy

Original: Network in Sec 3.3 that executes a program generation policy while automatically adapting to different grammars encoded in each task speciﬁcation.
Revised: Network in Sec 3.3 that executes a program generation policy while automatically adapting to different grammars encoded in each task speciﬁcation.

Original: 3. F ORMAL D EFINITIONS
Revised: 3. F ORMAL D EFINITIONS

Original: We formally deﬁne key concepts in the SyGuS problem formulation as follows.
Revised: We formally deﬁne key concepts in the SyGuS problem formulation as follows.

Original: semantic spec φ : The spec itself is a program written using some grammar.
Revised: semantic spec φ : The spec itself is a program written using some grammar.

Original: In our case, the grammar used in spec φ is different from the grammar G that speciﬁes the syntax of the output program.
Revised: In our case, the grammar used in spec φ is different from the grammar G that speciﬁes the syntax of the output program.

Original: However, in many practical cases the tokens (i.e., the dictionary of terminal symbols) may be shared across the input spec and the output program.
Revised: However, in many practical cases the tokens (i.e., the dictionary of terminal symbols) may be shared across the input spec and the output program.

Original: CFG G : A context free grammar (CFG) is deﬁned as G =
Revised: CFG G : A context free grammar (CFG) is deﬁned as G =

Original: (cid:104)V , Σ , R , s (cid:105) .
Revised: (cid:104)V , Σ , R , s (cid:105) .

Original: Here V denotes the nonterminal tokens, while Σ represents the terminal tokens.
Revised: Here V denotes the nonterminal tokens, while Σ represents the terminal tokens.

Original: s is a special token that denotes the start of the language, and the language is generated according to the production rules deﬁned in R .
Revised: s is a special token that denotes the start of the language, and the language is generated according to the production rules deﬁned in R .

Original: For a given non-terminal, the associated production rules can be written as α → β 1 | β 2 . . .
Revised: For a given non-terminal, the associated production rules can be written as α → β 1 | β 2 . . .

Original: | β n α , where n α is the branching factor for non-terminal α ∈ V , and β i = u 1 u 2 . . .
Revised: | β n α , where n α is the branching factor for non-terminal α ∈ V , and β i = u 1 u 2 . . .

Original: u | β i | ∈ ( V (cid:83) Σ) ∗ .
Revised: u | β i | ∈ ( V (cid:83) Σ) ∗ .

Original: Each production rule α → β i ∈ R represents a way of expanding the grammar tree, by attaching nodes u 1 , u 2 , . . .
Revised: Each production rule α → β i ∈ R represents a way of expanding the grammar tree, by attaching nodes u 1 , u 2 , . . .

Original: , u | β
Revised: , u | β

Original: i | to node α .
Revised: i | to node α .

Original: The expansion is repeated until all the leaf nodes are terminals.
Revised: The expansion is repeated until all the leaf nodes are terminals.

Original: Output function f : The output is a program in the language generated by G .
Revised: Output function f : The output is a program in the language generated by G .

Original: A valid output f must satisfy both the syntactic constraints speciﬁed by G and the semantic constraints speciﬁed by φ .
Revised: A valid output f must satisfy both the syntactic constraints speciﬁed by G and the semantic constraints speciﬁed by φ .

Original: 3. T ASK REPRESENTATION
Revised: 3. T ASK REPRESENTATION

Original: Different from traditional neural program synthesis tasks, where the program grammar and vocabulary is ﬁxed, each individual task in our setting has its own form of grammar and semantic speciﬁcation.
Revised: Different from traditional neural program synthesis tasks, where the program grammar and vocabulary is ﬁxed, each individual task in our setting has its own form of grammar and semantic speciﬁcation.

Original: Thus in the program generation phase (which we will elucidate in Sec 3.3), one cannot assume a ﬁxed CFG and use a tree decoder like in Kusner et al.
Revised: Thus in the program generation phase (which we will elucidate in Sec 3.3), one cannot assume a ﬁxed CFG and use a tree decoder like in Kusner et al.

Original: (2017) and Bunel et al.
Revised: (2017) and Bunel et al.

Original: To enable such generalization across different grammars, the information about the CFG for each task needs to be captured in the task representation.
Revised: To enable such generalization across different grammars, the information about the CFG for each task needs to be captured in the task representation.

Original: Since the semantic spec program φ and the CFG G have rich structural information, it is natural to use graphs for their representation.
Revised: Since the semantic spec program φ and the CFG G have rich structural information, it is natural to use graphs for their representation.

Original: Representing the programs using graphs has been successfully used in many programming language domains.
Revised: Representing the programs using graphs has been successfully used in many programming language domains.

Original: In our work, we further extend the approach by Allamanis et al.
Revised: In our work, we further extend the approach by Allamanis et al.

Original: (2018b) with respect to the following aspects:
Revised: (2018b) with respect to the following aspects:

Original: Logical Spec ! "

Original: X OR Y AND Z

Original: Grammar # " s -> d 1 OR d 1 | d 1 AND d 1 d 1 -> X OR Y | d 2 AND dd 2 -> ......

Original: Task

Original: Logical Spec !

Original: $ X XOR Y

Original: OR (X AND Z)

Original: Grammar # $ s -> d 1 XOR d 1 d 1 -> d 2 AND d 2 | d 2 OR d 2 | d 2 XOR dd 2 -> ......

Original: Task•

Original: Global graph embedding: • Embedding of production rules d 1 -> X OR Y | d 2 AND dd 1 _OR d 1 _

Original: AND ==

Original: > ℎ(' " ) ......

Original: [Figure]

Original: [Figure]

Original: embedding: •

Original: Embedding

Original: [Figure]

Original: d 1 _AND d 1 _OR ==> ℎ(' $ ) ......

Original: GRAPH EMBED d 1 -> d 2 AND d 2 | d 2 OR d 2 | d 2 XOR dd 1 _XOR

Original: S d

Original: X d

Original: Y OR ? )

Original: (*+") d 1 _OR d 1 _

Original: AND

Original: Context vector , *

Original: Embedding matrix - . / ⊙ 1 2 * ℎ(' " ,) (*+") )

Original: d~ d

Original: OR d 2 AND

Original: S dd

Original: AND ? )

Original: (*+")

Original: Context vector , *

Original: XOR dd

Original: Embedding

Original: [Figure]

Original: - . / ⊙ d 1 _AND d 1 _OR d 1 _XOR 1 2 * ℎ(' $ ,) (*+") )

Original: ~

Original: [Figure] XOR dd [Figure] Instead of only representing the semantic spec program φ as a graph, we propose to jointly represent it along with the grammar G .
Revised: • Instead of only representing the semantic spec program φ as a graph, we propose to jointly represent it along with the grammar G .

Original: • To allow information exchange between the two graphs, we leverage the idea of Static Single Assignment (SSA) form in compiler design.
Revised: • To allow information exchange between the two graphs, we leverage the idea of Static Single Assignment (SSA) form in compiler design.

Original: That is, the same variable (token) that may be assigned (deﬁned) at many different places should be viewed differently, but on the other hand, these variations correspond to the same original thing.
Revised: That is, the same variable (token) that may be assigned (deﬁned) at many different places should be viewed differently, but on the other hand, these variations correspond to the same original thing.

Original: Speciﬁcally, we introduce global nodes for shared tokens and global links connecting these globally shared nodes and local nodes that (re)deﬁne corresponding tokens.
Revised: Speciﬁcally, we introduce global nodes for shared tokens and global links connecting these globally shared nodes and local nodes that (re)deﬁne corresponding tokens.

Original: The overall representation framework is described in Figure 1.
Revised: The overall representation framework is described in Figure 1.

Original: To construct the graph, we ﬁrst build the abstract syntax tree (AST) for the semantic spec program φ , according to its own grammar (typically different from the output grammar G ).
Revised: To construct the graph, we ﬁrst build the abstract syntax tree (AST) for the semantic spec program φ , according to its own grammar (typically different from the output grammar G ).

Original: To represent the grammar G , we associate each symbol in V (cid:83) Σ with a node representation.
Revised: To represent the grammar G , we associate each symbol in V (cid:83) Σ with a node representation.

Original: Furthermore, for a non-terminal α and its corresponding production rules α → β 1 | β 2 . . .
Revised: Furthermore, for a non-terminal α and its corresponding production rules α → β 1 | β 2 . . .

Original: | β n α , we create additional nodes α i for each substitute
Revised: | β n α , we create additional nodes α i for each substitute

Original: β i .
Revised: β i .

Original: The purpose is to enable grammar adaptive program generation, which we elaborate in Sec 3.3.
Revised: The purpose is to enable grammar adaptive program generation, which we elaborate in Sec 3.3.

Original: As a simpliﬁcation, we merge all nodes α i representing β
Revised: As a simpliﬁcation, we merge all nodes α i representing β

Original: i that is a single terminal token into one node.
Revised: i that is a single terminal token into one node.

Original: Finally, the global nodes for shared tokens in Σ are created to link together the shared variable and operator nodes.
Revised: Finally, the global nodes for shared tokens in Σ are created to link together the shared variable and operator nodes.

Original: This enables information exchange between the syntactic and semantics speciﬁcations.
Revised: This enables information exchange between the syntactic and semantics speciﬁcations.

Original: To encode the joint graph G ( φ, G ) , we use graph neural networks to get the vector representation for each node in the graph.
Revised: To encode the joint graph G ( φ, G ) , we use graph neural networks to get the vector representation for each node in the graph.

Original: Speciﬁcally, for each node v ∈ G , we use the following parameterization for one step of message passing style update:
Revised: Speciﬁcally, for each node v ∈ G , we use the following parameterization for one step of message passing style update:

Original: [Equation]
Revised: [Equation]

Original: Lastly, { h Tv } v ∈G , h Tv ∈ R
Revised: Lastly, { h Tv } v ∈G , h Tv ∈ R

Original: d are the set of node embeddings.
Revised: d are the set of node embeddings.

Original: Here N ( v ) is the set of neighbor nodes of v , and e u,v denotes the type of edge that links the node u and v .
Revised: Here N ( v ) is the set of neighbor nodes of v , and e u,v denotes the type of edge that links the node u and v .

Original: We parameterize F in a way similar to GGNN (Li et al., 2015), i.e., F ( h t , e ) =
Revised: We parameterize F in a way similar to GGNN (Li et al., 2015), i.e., F ( h t , e ) =

Original: σ ( W e (cid:62) t h t ) where we use different matrices
Revised: σ ( W e (cid:62) t h t ) where we use different matrices

Original: W ∈ R d × d for different edge types and different propagation steps t .
Revised: W ∈ R d × d for different edge types and different propagation steps t .

Original: We sum over all the node embeddings to get the global graph embedding h ( G ) .
Revised: We sum over all the node embeddings to get the global graph embedding h ( G ) .

Original: In addition to the node embeddings and global graph embedding, we also obtain the embedding matrix for each non-terminal node.
Revised: In addition to the node embeddings and global graph embedding, we also obtain the embedding matrix for each non-terminal node.

Original: Speciﬁcally, given node α , we will have the embedding matrix
Revised: Speciﬁcally, given node α , we will have the embedding matrix

Original: H α ∈ R n α × d , where the i th row of H ( i )
Revised: H α ∈ R n α × d , where the i th row of H ( i )

Original: α is the embedding of node α i that corresponds to substitution β i .
Revised: α is the embedding of node α i that corresponds to substitution β i .

Original: This enables the grammar adaptive tree expansion in Sec 3.3.
Revised: This enables the grammar adaptive tree expansion in Sec 3.3.

Original: 3. G RAMMAR A DAPTIVE P OLICY N ETWORK
Revised: 3. G RAMMAR A DAPTIVE P OLICY N ETWORK

Original: To enable the meta-solver to generalize across different tasks, both the task representation and program generation policy should be shared.
Revised: To enable the meta-solver to generalize across different tasks, both the task representation and program generation policy should be shared.

Original: We perform task conditional program generation for this purpose.
Revised: We perform task conditional program generation for this purpose.

Original: Overall the generation is implemented using tree recursive generation, in the depth-ﬁrst search (DFS) order.
Revised: Overall the generation is implemented using tree recursive generation, in the depth-ﬁrst search (DFS) order.

Original: However, to handle different grammars speciﬁed in each task, we propose to use the grammar adaptive policy network.
Revised: However, to handle different grammars speciﬁed in each task, we propose to use the grammar adaptive policy network.

Original: The key idea is to make the policy parameterized by decision embedding, rather than a ﬁxed set of parameters.
Revised: The key idea is to make the policy parameterized by decision embedding, rather than a ﬁxed set of parameters.

Original: This mechanism is inspired by the pointer network (Vinyals et al., 2015) and graph algorithm learning
Revised: This mechanism is inspired by the pointer network (Vinyals et al., 2015) and graph algorithm learning

Original: (Dai et al., 2017).
Revised: (Dai et al., 2017).

Original: Speciﬁcally, suppose we are at the decision step t and try to expand the non-terminal node α t .
Revised: Speciﬁcally, suppose we are at the decision step t and try to expand the non-terminal node α t .

Original: For different tasks, the non-terminals may not be the same; furthermore, the number of ways to expand a certain non-terminal can also be different.
Revised: For different tasks, the non-terminals may not be the same; furthermore, the number of ways to expand a certain non-terminal can also be different.

Original: As a result, we cannot simply have a parameterized layer W (cid:62) h
Revised: As a result, we cannot simply have a parameterized layer W (cid:62) h

Original: α t to calculate the logits of multinomial distribution.
Revised: α t to calculate the logits of multinomial distribution.

Original: Rather, we use the embedding matrix H α t ∈ R n αt ×
Revised: Rather, we use the embedding matrix H α t ∈ R n αt ×

Original: d to perform decision for this time step.
Revised: d to perform decision for this time step.

Original: This embedding matrix is obtained as described in Sec 3.2.
Revised: This embedding matrix is obtained as described in Sec 3.2.

Revised: [Figure]

Revised: # " s -> d 1 OR d 1 | d 1 AND d 1 d 1 -> X OR Y | d

Original: Now we are able to build our policy network in an auto-regressive way.
Revised: [Figure] dd 2 -> ...... Task Logical Spec !

Revised: $ X XOR Y

Revised: OR (X AND Z)

Revised: [Figure]

Revised: # $ s -> d 1 XOR d 1 d 1 -> d 2 AND d 2 | d 2 OR d 2 | d 2 XOR dd 2 -> ......

Revised: [Figure]

Revised: [Figure]

Revised: d 1 -> X OR Y | d

Revised: [Figure]

Revised: dd 1 _

Revised: [Figure]

Revised: d

Revised: [Figure]

Revised: ℎ('

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: $

Revised: [Figure]

Revised: [Figure]

Revised: |

Revised: [Figure]

Revised: S d

Revised: X d

Revised: [Figure]

Revised: [Figure]

Revised: d

Revised: [Figure]

Revised: [Figure]

Revised: ℎ(

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: Now we are able to build our policy network in an auto-regressive way.

Original: Speciﬁcally, the policy π ( f | φ, G ) can be parameterized as:
Revised: Speciﬁcally, the policy π ( f | φ, G ) can be parameterized as:

Original: [Equation]
Revised: [Equation]

Original: G ) , T
Revised: , G

Original: [Equation]
Revised: [Equation]

Original:  where T ( t − 1) =
Revised: f | (cid:89) t = π ( a t | h ( G ) , T ( t − 1) ) , where T ( t − 1) =

Original: α 1 . . .
Revised: α 1 . . .

Original: α t − 1 denotes the partial tree (2)
Revised: α t − 1 denotes the partial tree (2)

Original: Here the probability of each action (in other words, each tree expansion decision) is deﬁned as π ( a t | h ( G ) , T ( t − 1) )
Revised: Here the probability of each action (in other words, each tree expansion decision) is deﬁned as π ( a t | h ( G ) , T ( t − 1) )

Original: ∝ exp( H ( i ) (cid:62) α s t ) , where s t ∈ R d is the context vector that captures the state of h ( G ) and T ( t − 1) .
Revised: ∝ exp( H ( i ) (cid:62) α s t ) , where s t ∈ R d is the context vector that captures the state of h ( G ) and T ( t − 1) .

Original: In our implementation, s t is tracked by a LSTM decoder whose hidden state is updated by the embedding of the chosen action h α t .
Revised: In our implementation, s t is tracked by a LSTM decoder whose hidden state is updated by the embedding of the chosen action h α t .

Original: The initial state s 0 is obtained by passing graph embedding h ( G ) through a dense layer with matching size.
Revised: The initial state s 0 is obtained by passing graph embedding h ( G ) through a dense layer with matching size.

Original:  S OLVING VIA R EINFORCEMENT L EARNING
Revised:  S OLVING VIA R EINFORCEMENT L EARNING

Original: In this section, we present a reinforcement learning framework for the meta-solver.
Revised: In this section, we present a reinforcement learning framework for the meta-solver.

Original: Formally, let θ denote the parameters of graph embedding and adaptive policy network.
Revised: Formally, let θ denote the parameters of graph embedding and adaptive policy network.

Original: For a given pair of instances ( φ, G ) , we learn a policy π θ ( f | φ, G ) parameterized by θ that generates f such that φ ≡ f .
Revised: For a given pair of instances ( φ, G ) , we learn a policy π θ ( f | φ, G ) parameterized by θ that generates f such that φ ≡ f .

Original: Reward design : The RL episode starts by accepting the representation of tuple (cid:104) φ, G (cid:105) as initial observation.
Revised: Reward design : The RL episode starts by accepting the representation of tuple (cid:104) φ, G (cid:105) as initial observation.

Original: During the episode, the model executes a sequence of actions to expand non-terminals in f , and ﬁnishes the episode when f is complete.
Revised: During the episode, the model executes a sequence of actions to expand non-terminals in f , and ﬁnishes the episode when f is complete.

Original: Upon ﬁnishing, the SAT solver is invoked and will return a binary ﬂag indicating whether f satisﬁes φ or not.
Revised: Upon ﬁnishing, the SAT solver is invoked and will return a binary ﬂag indicating whether f satisﬁes φ or not.

Original: An obvious reward design would be directly using the binary value as the episode return.
Revised: An obvious reward design would be directly using the binary value as the episode return.

Original: However, this leads to a high variance in returns as well as a highly non-smooth loss surface.
Revised: However, this leads to a high variance in returns as well as a highly non-smooth loss surface.

Original: Here, we propose to smooth the reward as follows: for each speciﬁcation φ we maintain a test case buffer B φ that stores all input examples observed so far.
Revised: Here, we propose to smooth the reward as follows: for each speciﬁcation φ we maintain a test case buffer B φ that stores all input examples observed so far.

Original: Each time the SAT solver is invoked for φ , if f passes then a full reward of 1 is given, otherwise the solver will generate a counter-example b besides the binary ﬂag.
Revised: Each time the SAT solver is invoked for φ , if f passes then a full reward of 1 is given, otherwise the solver will generate a counter-example b besides the binary ﬂag.

Original: We then sample interpolated examples around b which we denote the set as ˆ B b .
Revised: We then sample interpolated examples around b which we denote the set as ˆ B b .

Original: Then the reward is given as the fractions of examples in B φ and ˆ B b where f has the equivalent output as φ
Revised: Then the reward is given as the fractions of examples in B φ and ˆ B b where f has the equivalent output as φ

Original: [Equation]
Revised: [Equation]

Original: At the end of the episode, the buffer is updated as B φ ← B φ ∪ ˆ B b for next time usage.
Revised: At the end of the episode, the buffer is updated as B φ ← B φ ∪ ˆ B b for next time usage.

Original: In the extreme case where all inputs can be enumerated, e.g. binary or discrete values, it reduces to computing the fraction of passed examples over the entire test case set.
Revised: In the extreme case where all inputs can be enumerated, e.g. binary or discrete values, it reduces to computing the fraction of passed examples over the entire test case set.

Original: This is implemented in our experiment on the cryptographic circuit synthesis task.
Revised: This is implemented in our experiment on the cryptographic circuit synthesis task.

Original: 4. L EARNING THE M ETA - SOLVER
Revised: 4. L EARNING THE M ETA - SOLVER

Original: In the meta-learning setting, the framework learns to represent a set of different programs and navigate the generation process under different constraints.
Revised: In the meta-learning setting, the framework learns to represent a set of different programs and navigate the generation process under different constraints.

Original: We utilize the Advantage Actor-Critic (A2C) for model training.
Revised: We utilize the Advantage Actor-Critic (A2C) for model training.

Original: Given a training set D , a minibatch of instances are sampled from D for each epoch.
Revised: Given a training set D , a minibatch of instances are sampled from D for each epoch.

Original: For each instance (cid:104) φ
Revised: For each instance (cid:104) φ

Original: i , G i (cid:105) , the model performs a complete rollout using policy π θ ( f | φ i , G i ) .
Revised: i , G i (cid:105) , the model performs a complete rollout using policy π θ ( f | φ i , G i ) .

Original: The actor-critic method computes the gradients w.r.t to θ of each instance as 
Revised: The actor-critic method computes the gradients w.r.t to θ of each instance as dθ ← | f | (cid:88) t = ∇

Original: [Equation]
Revised: [Equation]

Original:  where γ denotes the discounting factor and V ( s t ; ω ) is a state value estimator parameterized by ω .
Revised: logπ ( α t | h ( G ) , T ( t ) ) ( γ | f |− t r − V ( s t ; ω )) , where γ denotes the discounting factor and V ( s t ; ω ) is a state value estimator parameterized by ω .

Original: In our implementation, this is modeled as a standard MLP with scalar output.
Revised: In our implementation, this is modeled as a standard MLP with scalar output.

Original: It is learned to ﬁt the expected return, i.e., min ω E (cid:13)(cid:13)(cid:13)(cid:80) | f | t =1 γ t r − V ( s t ; ω ) (cid:13)(cid:13)(cid:13) .
Revised: It is learned to ﬁt the expected return, i.e., min ω E (cid:13)(cid:13)(cid:13)(cid:80) | f | t =1 γ t r − V ( s t ; ω ) (cid:13)(cid:13)(cid:13) .

Original: Gradients obtained from each instance are averaged over the minibatch before applying to the parameter.
Revised: Gradients obtained from each instance are averaged over the minibatch before applying to the parameter.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: [Figure]
Revised: [Figure]

Original: We evaluate the our framework on cryptographic circuit synthesis tasks (Eldib et al., 2016) which constitute a challenging benchmark suite from the general track of the SyGuS Competition (2017).
Revised: We evaluate the our framework on cryptographic circuit synthesis tasks (Eldib et al., 2016) which constitute a challenging benchmark suite from the general track of the SyGuS Competition (2017).

Original: The dataset contains 214 tasks, each of which is a pair of logical speciﬁcation, describing the correct functionality, and a context free grammar, describing the timing constraints for input signals.
Revised: The dataset contains 214 tasks, each of which is a pair of logical speciﬁcation, describing the correct functionality, and a context free grammar, describing the timing constraints for input signals.

Original: The goal is to ﬁnd an equivalent logical expression which is required to follow the given context free grammar in order to avoid potential timing channel vulnerabilities.
Revised: The goal is to ﬁnd an equivalent logical expression which is required to follow the given context free grammar in order to avoid potential timing channel vulnerabilities.

Original: Figure 3 shows an illustrative example.
Revised: Figure 3 shows an illustrative example.

Original: Each synthesis task has a different logical speciﬁcation as well as timing constraints, and both the logical speciﬁcation and context free grammar varies from task to task, posing a signiﬁcant challenge in representation learning.
Revised: Each synthesis task has a different logical speciﬁcation as well as timing constraints, and both the logical speciﬁcation and context free grammar varies from task to task, posing a signiﬁcant challenge in representation learning.

Original: As a result, this suite of tasks serves as an ideal testbed for our learning framework and its capability to generalize to unseen speciﬁcations and grammars.
Revised: As a result, this suite of tasks serves as an ideal testbed for our learning framework and its capability to generalize to unseen speciﬁcations and grammars.

Original: The experiments are conducted in two learning settings.
Revised: The experiments are conducted in two learning settings.

Original: First, we test our framework as an outof-box solver, which means the training set D and testing set D (cid:48) are the same and contain only one instance.
Revised: First, we test our framework as an outof-box solver, which means the training set D and testing set D (cid:48) are the same and contain only one instance.

Original: In other words, the framework is tasked to solve only one instance at a time.
Revised: In other words, the framework is tasked to solve only one instance at a time.

Original: This test-on-train setting serves to investigate the capacity of our framework in representation and policy learning, as the model can arbitrarily “exploit” the problem without worrying about overﬁtting.
Revised: This test-on-train setting serves to investigate the capacity of our framework in representation and policy learning, as the model can arbitrarily “exploit” the problem without worrying about overﬁtting.

Original: This setting also enables us to compare our framework to classical solvers developed in the formal methods community.
Revised: This setting also enables us to compare our framework to classical solvers developed in the formal methods community.

Original: As those solvers do not utilize learning-based strategies, it is sensible to also limit our framework not to carry over prior knowledge from a separate training set.
Revised: As those solvers do not utilize learning-based strategies, it is sensible to also limit our framework not to carry over prior knowledge from a separate training set.

Original: Second, we evaluate the model as a meta-solver which is trained over a training set D , and ﬁnetuned on each of the new tasks in a separate set D (cid:48) .
Revised: Second, we evaluate the model as a meta-solver which is trained over a training set D , and ﬁnetuned on each of the new tasks in a separate set D (cid:48) .

Original: In this setting, we aim to demonstrate that our framework is capable of learning a transferable representation and policy in order to efﬁciently adapt to unseen tasks. 
Revised: In this setting, we aim to demonstrate that our framework is capable of learning a transferable representation and policy in order to efﬁciently adapt to unseen tasks. [Table]

Original: 5. L EARNING AN O UT - OF -B OX S OLVER
Revised: 5. L EARNING AN O UT - OF -B OX S OLVER

Original: In the out-of-box solver setting, we compare our framework against solvers built based on two classical approaches: a SAT/SMT constraint solving based approach and a search based approach.
Revised: In the out-of-box solver setting, we compare our framework against solvers built based on two classical approaches: a SAT/SMT constraint solving based approach and a search based approach.

Original: For the former, we choose CVC4 (Reynolds et al., 2015), which is the state-of-the-art SMT constraint solver; for the latter, we choose EUSolver (Alur et al., 2017b), which is the winner of the SyGuS 2017 Competition (Alur et al., 2017a).
Revised: For the former, we choose CVC4 (Reynolds et al., 2015), which is the state-of-the-art SMT constraint solver; for the latter, we choose EUSolver (Alur et al., 2017b), which is the winner of the SyGuS 2017 Competition (Alur et al., 2017a).

Original: Furthermore, we build a search based solver as baseline, ESymbolic, which systematically expands non-terminals in a predeﬁned order (e.g. depth-ﬁrst-search) and effectively prunes away partially generated candidates by reducing it to 2QBF (Balabanov et al., 2016)[Table] satisﬁability check.
Revised: Furthermore, we build a search based solver as baseline, ESymbolic, which systematically expands non-terminals in a predeﬁned order (e.g. depth-ﬁrst-search) and effectively prunes away partially generated candidates by reducing it to 2QBF (Balabanov et al., 2016) satisﬁability check.

Original: ESymbolic can be viewed as a generalization of EUSolver by replacing the carefully designed domain-speciﬁc heuristics (e.g. indistinguishability and uniﬁcation) with 2QBF.
Revised: ESymbolic can be viewed as a generalization of EUSolver by replacing the carefully designed domain-speciﬁc heuristics (e.g. indistinguishability and uniﬁcation) with 2QBF.

Original: In order to make the comparison fair, we run all solvers on the same platform with a single core
Revised: In order to make the comparison fair, we run all solvers on the same platform with a single core

Original: CPU available 1 , even though our framework could take advantage of hardware accelerations, for instance, via GPUs and TPUs.
Revised: CPU available 1 , even though our framework could take advantage of hardware accelerations, for instance, via GPUs and TPUs.

Original: We measure the performance of each solver by counting the number of instances it can solve given a 6 hours limit spent on each task.
Revised: We measure the performance of each solver by counting the number of instances it can solve given a 6 hours limit spent on each task.

Original: It is worth noting that comparing running time only gives a limited view of the solvers’ performance.
Revised: It is worth noting that comparing running time only gives a limited view of the solvers’ performance.

Original: Although the hardware is the same, the software implementation can make many differences.
Revised: Although the hardware is the same, the software implementation can make many differences.

Original: For instance, CVC4 is carefully redesigned and re-implemented in C++ as the successor of CVC3 (Barrett et al., 2003), which has been actively improved for more than a decade.
Revised: For instance, CVC4 is carefully redesigned and re-implemented in C++ as the successor of CVC3 (Barrett et al., 2003), which has been actively improved for more than a decade.

Original: To our best knowledge, the design and implementation of EUSolver is directly guided by and heavily tuned according to SyGuS benchmarks.
Revised: To our best knowledge, the design and implementation of EUSolver is directly guided by and heavily tuned according to SyGuS benchmarks.

Original: In contrast, our framework is a proof-of-concept prototype implemented in Python and has not yet been tuned for running time performance.
Revised: In contrast, our framework is a proof-of-concept prototype implemented in Python and has not yet been tuned for running time performance.

Original: In Table 1, we summarize the total number of instances solved by each solver as well as the maximum, average and median running time spent on solved instances.
Revised: In Table 1, we summarize the total number of instances solved by each solver as well as the maximum, average and median running time spent on solved instances.

Original: In terms of the absolute number of solved instances, our framework is not yet as good as EUSolver, which is equipped with specialized heuristics.
Revised: In terms of the absolute number of solved instances, our framework is not yet as good as EUSolver, which is equipped with specialized heuristics.

Original: However, EUSolver fails to solve 4 instances that are only solved by our framework.
Revised: However, EUSolver fails to solve 4 instances that are only solved by our framework.

Original: All instances solved by CVC4 and ESymbolic are a strict subset of instances solved by EUSolver.
Revised: All instances solved by CVC4 and ESymbolic are a strict subset of instances solved by EUSolver.

Original: Thus, besides being a new promising approach, our framework already plays a supplementary role for improving the current state-of-the-art.
Revised: Thus, besides being a new promising approach, our framework already plays a supplementary role for improving the current state-of-the-art.

Original: Compared with the state-of-the-art CVC solver, our framework has smaller maximum time but higher average and median time usage.
Revised: Compared with the state-of-the-art CVC solver, our framework has smaller maximum time but higher average and median time usage.

Original: This suggests that our framework excels at solving difﬁcult instances with better efﬁciency.
Revised: This suggests that our framework excels at solving difﬁcult instances with better efﬁciency.

Original: This observation is further conﬁrmed in Figure 4, where we plot the time usage along with the number of instances solved.
Revised: This observation is further conﬁrmed in Figure 4, where we plot the time usage along with the number of instances solved.

Original: This suggests that canonical solvers such as CVC4 are efﬁcient in solving simple instances, but have inferior scalability compared to our dynamically adapted approach when the problem becomes more difﬁcult, where we can see a steeper increase in time usages by CVC in solving 110 and more instances.
Revised: This suggests that canonical solvers such as CVC4 are efﬁcient in solving simple instances, but have inferior scalability compared to our dynamically adapted approach when the problem becomes more difﬁcult, where we can see a steeper increase in time usages by CVC in solving 110 and more instances.

Original: Though EUSolver has superior scalability, it is achieved by a number of heuristics that are manually designed and iteratively improved by experts with the same benchmark on hand.
Revised: Though EUSolver has superior scalability, it is achieved by a number of heuristics that are manually designed and iteratively improved by experts with the same [Figure] Out-of-Box Solver 18K 102K Meta-Solver 4.5K 59K

Revised: Reduction 12.5 × 4.0 × 1.7 ×

Original:  In contrast, our framework learns a policy to solve hard instances from scratch on the ﬂy without requiring training data at all.
Revised: [Figure] benchmark on hand. In contrast, our framework learns a policy to solve hard instances from scratch on the ﬂy without requiring training data at all.

Original: 5. L EARNING ACROSS DIFFERENT TASKS
Revised: 5. L EARNING ACROSS DIFFERENT TASKS

Original: We next evaluate whether our framework is capable of learning transferable knowledge across different synthesis tasks.
Revised: We next evaluate whether our framework is capable of learning transferable knowledge across different synthesis tasks.

Original: We randomly split the 214 circuits synthesis tasks into two sets: 150 tasks for training and the rest 64 tasks for testing.
Revised: We randomly split the 214 circuits synthesis tasks into two sets: 150 tasks for training and the rest 64 tasks for testing.

Original: The meta-solver is then trained on the training set forepochs using methods introduced in Sec 4.1.
Revised: The meta-solver is then trained on the training set forepochs using methods introduced in Sec 4.1.

Original: For each epoch, a batch of 10 tasks are sampled.
Revised: For each epoch, a batch of 10 tasks are sampled.

Original: The gradients of each task are averaged and applied to the model parameters using Adam optimizer.
Revised: The gradients of each task are averaged and applied to the model parameters using Adam optimizer.

Original: In testing phase, the trained meta-solver is ﬁnetuned on each task in the testing set until either a correct [Figure] Out-of-Box Solver 18K 102K Meta-Solver 4.5K 59K
Revised: In testing phase, the trained meta-solver is ﬁnetuned on each task in the testing set until either a correct program is synthesized or timeout occurs.

Original: Reduction 12.5 × 4.0 × 1.7 ×

Original: [Figure]

Original: program is synthesized or timeout occurs.

Original: This process is similar to the setting in Sec 5.1 but with smaller learning  and exploration.
Revised: This process is similar to the setting in Sec 5.1 but with smaller learning rate and exploration.

Original: We compare the trained meta-solver with the out-of-box solver in solving tasks in the test set.
Revised: We compare the trained meta-solver with the out-of-box solver in solving tasks in the test set.

Original: Out of 64 testing tasks, the out-of-box solver and meta-solver can solve 36 and 37 tasks, respectively.
Revised: Out of 64 testing tasks, the out-of-box solver and meta-solver can solve 36 and 37 tasks, respectively.

Original: Besides the additional task solved, the performance is also greatly improved by meta-solver, which is shown in Figure 5.
Revised: Besides the additional task solved, the performance is also greatly improved by meta-solver, which is shown in Figure 5.

Original: Table 5(a) shows the accumulated number of candidates generated to successfully solve various ratios of testing tasks.
Revised: Table 5(a) shows the accumulated number of candidates generated to successfully solve various ratios of testing tasks.

Original: We see that the number of explored candidates by meta-solver is signiﬁcantly reduced: for 40% of testing tasks (i.e., 66% of solved tasks), meta-learning enable 4x reduction on average.
Revised: We see that the number of explored candidates by meta-solver is signiﬁcantly reduced: for 40% of testing tasks (i.e., 66% of solved tasks), meta-learning enable 4x reduction on average.

Original: The accumulated reduction for all solved tasks (60% of testing tasks) is not that signiﬁcant.
Revised: The accumulated reduction for all solved tasks (60% of testing tasks) is not that signiﬁcant.

Original: This is because meta-learning improve dramatically for most (relatively) easy tasks but helps slightly for a few hard tasks, which actually dominate the number of generated candidates.
Revised: This is because meta-learning improve dramatically for most (relatively) easy tasks but helps slightly for a few hard tasks, which actually dominate the number of generated candidates.

Original: Figure 5(b) shows the speedup distribution over the 36 commonly solved tasks.
Revised: Figure 5(b) shows the speedup distribution over the 36 commonly solved tasks.

Original: Meta-solver achieves at least 2x speedup for most benchmarks, orders of magnitude improvement for 10 out of 36 unseen tasks, and solves one task that is not solvable without meta-learning.
Revised: Meta-solver achieves at least 2x speedup for most benchmarks, orders of magnitude improvement for 10 out of 36 unseen tasks, and solves one task that is not solvable without meta-learning.

Original:  RELATED WORK
Revised:  RELATED WORK

Original: We survey work on symbolic program synthesis, neural program synthesis, and neural induction.
Revised: We survey work on symbolic program synthesis, neural program synthesis, and neural induction.

Original: Symbolic program synthesis.
Revised: Symbolic program synthesis.

Original: Automatically synthesizing a program from its speciﬁcation was ﬁrst posed by Manna & Waldinger (1971).
Revised: Automatically synthesizing a program from its speciﬁcation was ﬁrst posed by Manna & Waldinger (1971).

Original: It received renewed attention with advances in SAT and SMT solvers (Solar-Lezama et al., 2006) and found application in problems in various domains as surveyed by Gulwani et al.
Revised: It received renewed attention with advances in SAT and SMT solvers (Solar-Lezama et al., 2006) and found application in problems in various domains as surveyed by Gulwani et al.

Original: In this context, SyGuS (Alur et al., 2013) was proposed as a common format to express these problems.
Revised: In this context, SyGuS (Alur et al., 2013) was proposed as a common format to express these problems.

Original: Several implementations of SyGuS solvers exist, including by constraint solving (Reynolds et al., 2015), divide-and-conquer (Alur et al., 2017b), and stochastic MCMC search (Schkufza et al., 2013), in addition to various domain-speciﬁc algorithms.
Revised: Several implementations of SyGuS solvers exist, including by constraint solving (Reynolds et al., 2015), divide-and-conquer (Alur et al., 2017b), and stochastic MCMC search (Schkufza et al., 2013), in addition to various domain-speciﬁc algorithms.

Original: A number of probabilistic techniques have been proposed to accelerate these solvers by modeling syntactic aspects of programs.
Revised: A number of probabilistic techniques have been proposed to accelerate these solvers by modeling syntactic aspects of programs.

Original: These include PHOG (Lee et al., 2018), log-bilinear tree-traversal models (Maddison & Tarlow, 2014), and graph-based statistical models (Nguyen & Nguyen, 2015).
Revised: These include PHOG (Lee et al., 2018), log-bilinear tree-traversal models (Maddison & Tarlow, 2014), and graph-based statistical models (Nguyen & Nguyen, 2015).

Original: Neural program synthesis.
Revised: Neural program synthesis.

Original: Several recent works have used neural networks to accelerate the discovery of desired programs.
Revised: Several recent works have used neural networks to accelerate the discovery of desired programs.

Original: These include DeepCoder (Balog et al., 2017), Bayou (Murali et al., 2018), RobustFill (Devlin et al., 2017), Differentiable FORTH
Revised: These include DeepCoder (Balog et al., 2017), Bayou (Murali et al., 2018), RobustFill (Devlin et al., 2017), Differentiable FORTH

Original: (Boˇsnjak et al., 2017), neurosymbolic program synthesis (Parisotto et al., 2016; Bunel et al., 2018), neural-guided deductive search (Vijayakumar et al., 2018), learning context-free parsers (Chen et al., 2018), and learning program invariants (Si et al., 2018).
Revised: (Boˇsnjak et al., 2017), neurosymbolic program synthesis (Parisotto et al., 2016; Bunel et al., 2018), neural-guided deductive search (Vijayakumar et al., 2018), learning context-free parsers (Chen et al., 2018), and learning program invariants (Si et al., 2018).

Original: The syntactic speciﬁcation in these approaches is ﬁxed by deﬁning a domain-speciﬁc language upfront.
Revised: The syntactic speciﬁcation in these approaches is ﬁxed by deﬁning a domain-speciﬁc language upfront.

Original: Also, with the exception of Si et al.
Revised: Also, with the exception of Si et al.

Original: , the semantic speciﬁcation takes the form of input-output examples.
Revised: , the semantic speciﬁcation takes the form of input-output examples.

Original: Broadly, these works have difﬁculty with symbolic constraints, and are primarily concerned with avoiding overﬁtting, coping with few examples, and tolerating noisy examples.
Revised: Broadly, these works have difﬁculty with symbolic constraints, and are primarily concerned with avoiding overﬁtting, coping with few examples, and tolerating noisy examples.

Original: Our work relaxes both these kinds of speciﬁcations to target the general SyGuS formulation.
Revised: Our work relaxes both these kinds of speciﬁcations to target the general SyGuS formulation.

Original: Recently Ellis et al.
Revised: Recently Ellis et al.

Original: (2018) propose gradually bootstrapping domain-speciﬁc languages for neurally-guided Bayesian program learning, while our work concerns learning programs that use similar grammars, which may or may not be incremental.
Revised: (2018) propose gradually bootstrapping domain-speciﬁc languages for neurally-guided Bayesian program learning, while our work concerns learning programs that use similar grammars, which may or may not be incremental.

Original: Neural program induction.
Revised: Neural program induction.

Original: Another body of work includes techniques in which the neural network is itself the computational substrate.
Revised: Another body of work includes techniques in which the neural network is itself the computational substrate.

Original: These include neural Turing machines (Graves et al., 2014) that can learn simple copying/sorting programs, the neural RAM model (Kurach et al., 2016) to learn pointer manipulation and dereferencing, the neural GPU model (Kaiser & Sutskever, 2015) to learn complex operations like binary multiplication, and Cai et al.
Revised: These include neural Turing machines (Graves et al., 2014) that can learn simple copying/sorting programs, the neural RAM model (Kurach et al., 2016) to learn pointer manipulation and dereferencing, the neural GPU model (Kaiser & Sutskever, 2015) to learn complex operations like binary multiplication, and Cai et al.

Original: (2017)’s work to incorporate recursion.
Revised: (2017)’s work to incorporate recursion.

Original: These approaches have fundamental problems regarding verifying and interpreting the output of neural networks.
Revised: These approaches have fundamental problems regarding verifying and interpreting the output of neural networks.

Original: In contrast, we propose tightly integrating a neural learner with a symbolic veriﬁer so that we obtain the scalability and ﬂexibility of neural learning and the correctness guarantees of symbolic veriﬁers.
Revised: In contrast, we propose tightly integrating a neural learner with a symbolic veriﬁer so that we obtain the scalability and ﬂexibility of neural learning and the correctness guarantees of symbolic veriﬁers.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We proposed a framework to learn a transferable representation and strategy in solving a general formulation of program synthesis, i.e. syntax-guided synthesis (SyGuS).
Revised: We proposed a framework to learn a transferable representation and strategy in solving a general formulation of program synthesis, i.e. syntax-guided synthesis (SyGuS).

Original: Compared to previous work on neural synthesis, our framework is capable of handling tasks where 1) the grammar and semantic speciﬁcation varies from task to task, and 2) the supervision is weak.
Revised: Compared to previous work on neural synthesis, our framework is capable of handling tasks where 1) the grammar and semantic speciﬁcation varies from task to task, and 2) the supervision is weak.

Original: Speciﬁcally, we introduced a graph neural network that can learn a joint representation over different pairs of syntactic and semantic speciﬁcations; we implemented a grammar adaptive network that enables program generation to be conditioned on the speciﬁc task; and ﬁnally, we proposed a meta-learning method based on the Advantage Actor-Critic (A2C) framework.
Revised: Speciﬁcally, we introduced a graph neural network that can learn a joint representation over different pairs of syntactic and semantic speciﬁcations; we implemented a grammar adaptive network that enables program generation to be conditioned on the speciﬁc task; and ﬁnally, we proposed a meta-learning method based on the Advantage Actor-Critic (A2C) framework.

Original: We compared our framework empirically against one baseline following a similar search fashion and two classical synthesis engines.
Revised: We compared our framework empirically against one baseline following a similar search fashion and two classical synthesis engines.

Original: Under the outof-box solver setting with limited computational resources and without any prior knowledge from training, our framework is able to solve 141 of 214 tasks, signiﬁcantly outperforming the baseline
Revised: Under the outof-box solver setting with limited computational resources and without any prior knowledge from training, our framework is able to solve 141 of 214 tasks, signiﬁcantly outperforming the baseline

Original: ESymbolic by 110.
Revised: ESymbolic by 110.

Original: In terms of the absolute number of solved tasks, the performance is comparable to two state-of-the-art solvers, CVC4 and EUSolver, which solve 129 and 153 respectively.
Revised: In terms of the absolute number of solved tasks, the performance is comparable to two state-of-the-art solvers, CVC4 and EUSolver, which solve 129 and 153 respectively.

Original: However, the two state-of-the-art solvers failed on 4 tasks solved by our framework.
Revised: However, the two state-of-the-art solvers failed on 4 tasks solved by our framework.

Original: When trained as a meta-solver, our framework is capable of accelerating the solving process by 2 × to 100 × . 
Revised: When trained as a meta-solver, our framework is capable of accelerating the solving process by 2 × to 100 × . A CKNOWLEDGMENTS

Revised: We thank the anonymous reviewers for their insightful comments.

Revised: This research was supported in part by NSF awards #1836936 and #