R EPRESENTING P ARTIAL P ROGRAMS WITH B LENDED A BSTRACT S EMANTICS
A BSTRACT
Synthesizing programs from examples requires searching over a vast, combinatorial space of possible programs.
In this search process, a key challenge is representing the behavior of a partially written program before it can be executed, to judge if it is on the right track and predict where to search next.
We introduce a general technique for representing partially written programs in a program synthesis engine.
We take inspiration from the technique of abstract interpretation, in which an approximate execution model is used to determine if an unﬁnished program will eventually satisfy a goal speciﬁcation.
Here we learn an approximate execution model implemented as a modular neural network.
By constructing compositional program representations that implicitly encode the interpretation semantics of the underlying programming language, we can represent partial programs using a ﬂexible combination of concrete execution state and learned neural representations, using the learned approximate semantics when concrete semantics are not known (in unﬁnished parts of the program).
We show that these hybrid neuro-symbolic representations enable execution-guided synthesizers to use more powerful language constructs, such as loops and higher-order functions, and can be used to synthesize programs more accurately for a given search budget than pure neural approaches in several domains.
I NTRODUCTION
Inductive program synthesis – the problem of inferring programs from examples – offers the promise of building machine learning systems which are interpretable, generalize quickly, and allow us to solve structured tasks such as planning and interacting with computer systems.
In recent years, neurally-guided program synthesis, which use deep learning to guide search over the space of possible programs, has emerged as a promising approach (Balog et al., 2016; Devlin et al., 2017).
In this framework, partially-constructed programs are judged to determine if they are on the right track and to predict where to search next (see Figure 1).
A key challenge in neural program synthesis is representing the behavior of partially written programs, in order to make these judgments.
In this work, we present a novel method for representing the semantic content of partially written code, which can be used to guide search to solve program synthesis tasks.
Recently, approaches which represent partial programs via their semantic state have been shown to be particularly effective.
In these execution-guided neural synthesis approaches (Chen et al., 2018; Ellis et al., 2019; Zohar & Wolf, 2018), partial programs are executed and represented with their return values.
However, execution is not always possible for a partial program.
In Figure 1, before the HOLE is ﬁlled with an integer value, we cannot meaningfully execute the partially-written loop in s .
This is a common problem for languages containing higher-order functions and control ﬂow, where execution of partially written code is often ill-deﬁned.
Thus, a key question is: How might we represent the semantics of unﬁnished code?
A classic method for representing program state, known as abstract interpretation, can be used to reason about the set of states that a partial program could reach, given the possible instantiations of the unﬁnished parts of the program.
Using abstract interpretation, an approximate execution model can determine if an unﬁnished program will eventually satisfy a goal speciﬁcation.
However, this
[Figure]
SEARCH
[Figure]
technique is often low-precision; hand-designed abstract execution models greatly overapproximate the set of possible execution states, and contain no notion of what code is likely to be written.
We hypothesize that, by mimicking the compositional structure of abstract interpretation, learned components can be used to effectively represent ambiguous program state.
In this work, we make two contributions: we introduce neural abstract semantics , in which a compositional, approximate execution model is used to represent partially written code.
We further introduce blended abstract semantics , which aims to represent the state of unﬁnished programs as faithfully as possible by concretely executing program components whenever possible, and otherwise, approximating program state with a learned abstract execution model.
This combination of learned execution and concrete execution allows robust representation of partial programs, which can be used for downstream synthesis tasks.
We show that our model can effectively learn to represent partial program states for languages where previous execution-guided synthesis techniques are not applicable.
In summary, • We introduce blended neural semantics, a novel method for representing the semantic state of partially written programs inspired by abstract interpretation.
• We describe how to integrate our program representations into existing approaches for learning search policies and search heuristics.
• We validate our new approach with program synthesis experiments in three domains: tower construction, list processing, and string editing.
We show that our approach outperforms neural synthesis baselines, solving at least 5% more programs in each domain.
B LENDED A BSTRACT S EMANTICS
Consider the problem of synthesizing arithmetic expressions from input–output pairs.
Suppose we have the following context-free grammar for expressions:
[Equation]
and a speciﬁcation X consisting of the input–output pairs { ( x = 3 , y = 7) , ( x = 5 , y = 11) } .
Suppose further that we have a candidate program (2 * x) + 1 ∈ G .
To check that this program is consistent with the speciﬁcation, we can evaluate it on the inputs x in the speciﬁcation according to the concrete semantics of the language.
The goal of synthesis is to ﬁnd an expression which satisﬁes the input-output examples under concrete semantics.
To ﬁnd such programs, we employ top-down search: starting with the top-level (incomplete) expression HOLE , we consider all possible expansions, ( HOLE → HOLE + HOLE , HOLE → 1 , HOLE → 2 , . . . ) and select the one we believe is most likely to succeed (Figure 1 left).
The more effectively we can ﬁlter the set of incomplete candidate programs, the faster our synthesis algorithm.
Conventional abstract interpretation solves this problem by deﬁning an alternative semantics for which even incomplete expressions can be evaluated.
However, constructing appropriate abstrac-
Neural abstract semantics
CONTEXT loop(4, seq(buildColumn(HOLE), HOLE))
loop(4, seq(buildColumn(1), moveHand(HOLE)))
loop(4, seq(buildColumn(1), HOLE)) ...
HOLE HOLE
HOLE HOLE SEARCH SPACE GOAL loopseq buildColumnmoveHand HOLE loopseq moveHand HOLE Blended abstract semantics loop 4 seq buildColumn
HOLE moveHand RNN representation (s, X) a; s, X) search state representation EMBED s
EMBED
EMBED f EMBED
EMBED EMBED
[Figure]
+ EMBED
[Figure]
x =
[Figure]
x =+
EMBED EMBED
[Figure]
tions is difﬁcult and requires domain-speciﬁc engineering; an ideal procedure would automatically discover an effective space of abstract interpretations.
Neural abstract semantics [[ · ]] nn
As a ﬁrst step, we implement the abstract interpretation procedure with a neural network.
(This is a natural choice: neural networks excel at representation learning, and the goal of abstract interpretation is to encode an informative representation of the set of values that could be returned by a partial program.)
For the program 1 + HOLE , we can encode the expression 1 to a learned representation (Figure 2a, top), likewise encode HOLE (Figure 2c), and ﬁnally employ a learned abstract implementation of the + operation (Figure 2b).
For concrete leaf nodes, such as constants or variables bound to constants, neural semantics are given using a state embedding function E MBED ( · ) , which maps any concrete state in the programming language into a vector representation: E MBED : ( State | R d ) → R d .
If the input to E MBED is already vector-valued, E MBED performs the identity operation.
Neural placeholders provide a method for computing a vector representation of unwritten code, denoted by the HOLE token.
To compute the representation for HOLE , we deﬁne a neural embedding function h which takes a context C and outputs a vector.
For each built-in function f , the neural abstract semantics of a function f are given by a separate neural module (a learned vector-valued function as in Andreas et al.
[[ f ]] nn with the same arity as f .
Therefore, computing the neural semantics means applying the neural function [[ f ]] nn to its arguments, which returns a vector.
Since the neural semantics mirrors the concrete semantics, its implementation does not require changes to the underlying language.
Blended abstract semantics [[ · ]] blend
Notice that for an expression such as (2 * x) + HOLE , the concrete value of the sub-expression (2 * x) is known, since it contains no holes.
The neural semantics above don’t make use of this knowledge.
To improve upon this, we extend neural semantics and introduce blended semantics, which alternates between neural and concrete interpretation as appropriate for a given expression:
• If the expression is a constant or a variable, use the concrete semantics.
• If the expression is a HOLE , use the neural semantics.
• If the expression is a function call, recursively evaluate the expressions that are the arguments to the function.
If all arguments evaluate to concrete values, execute the function concretely.
If any argument evaluates to a vector representation, transform all concrete values to vectors using E MBED and apply the neural semantics of the function.
Because blended abstract semantics replaces concrete sub-components with their concrete values, we expect blended semantics to result in more robust representations, especially for long or complex programs where large portions can be concretely executed.
See the appendix for more details.
Synthesis To perform synthesis, we experiment with methods to guide search introduced in Ellis et al.
In this work, the search over partial programs is formulated as an MDP, in which each state is a pair ( s, X ) consisting of a partial-program and a speciﬁcation, and actions a ∈ G are expansions of HOLE s under rules under the grammar.
We assume a reward of 1 for programs which satisfy X .
In this framework, we learn to search by training a policy π ( a | s, X ) that proposes expansions to s , and optionally a value function V ( s, X ) that predicts the probability that X is solvable via any expansion of s .
Details can be found in the appendix.
At test time, we explore a variety of code-writing search algorithms.
Using only a policy, we can employ sample-based search and best-ﬁrst search (where the log probability of generating s under π is used as the scoring function).
With the addition of a learned value function, we can perform A*-based search with − log V ( s, X ) as a heuristic (see Ellis et al.
(2019) for details).
E XPERIMENTS
We evaluate our model in two domains containing language constructs not handled by concrete execution-guided synthesis approaches: a tower-building domain with looping constructs, and a list-processing domain with higher order functions.
We additionally test on a string-editing domain for which execution-guided synthesis is possible, but requires extensive DSL modiﬁcation; there, we examine how our approach fares without these modiﬁcations (see appendix).
Looping constructs: Tower construction We begin by investigating how our model performs in generative programming domains with higher-level control ﬂow such as loops.
Looping programs are an essential part of sophisticated programming languages, and aren’t naturally handled by previous execution-guided synthesis approaches.
Our experiments in the tower-building domain employ a DSL similar to the language depicted in the introduction to construct towers in a 2D world.
As above, the goal is to construct a program which successfully renders to a target image (examples in
Figure 4).
Language details can be found in the appendix.
We compared against two baselines (see
Figure 1): (1) Neural abstract semantics (deﬁned in Section 2), which does not apply concrete execution to concrete subtrees, and (2) RNN encoding, which encodes partial programs using a GRU: π ( a | GRU enc ( s ) , X ) .
To evaluate our model, we constructed a test set of tower-building problems involving combinations of tower-building motifs seen during training.
We evaluate our models by performing best-ﬁrst search from the learned policy.
We also test using a value function, where we are doing A* search with the policy as the prior cost and the value function as the heuristic future cost estimate.
Figure 3 left shows our overall synthesis results in the tower-building domain, measuring the percentage of test problems solved as a function of the number of search nodes (partial programs) considered.
The sequence encoding performs poorly and is unable to solve a majority of test problems.
The neural abstract semantics model achieves better performance, solving about half of the test problems within the allotted search budget.
Blended execution outperforms both baselines.
We additionally observe that adding a value function as a search heuristic further increases performance of our blended model, which is consistent with the ﬁndings in Ellis et al.
Higher-order functions: functional list processing
In our second experimental domain, we seek to answer two questions: How well does our model perform on input-output synthesis?
How effectively can it synthesize programs containing higher-order functions?
Although previous work (Zohar & Wolf, 2018) has successfully applied execution-guided approaches to list processing (using the DeepCoder language), the use of higher-order functions was severely limited: only a small, predeﬁned set of “lambdas,” (such as (*2) , is even , (>0) ) were used as arguments for higherorder functions.
For example, synthesizing a program which “ﬁlters all elements divisible byfrom a list” is not possible with this DSL.
However, in real programming languages, higher-order functions must be able to accept a combinatorially large set of possible lambda functions as input.
This presents a challenge for execution-guided synthesis approaches such as Zohar & Wolf (2018), for which the assumption of a small set of lambda functions is key.
To this end, we modiﬁed the
Deepcoder DSL to allow a richer set of possible programs.
We replaced the predeﬁned set of lambda functions with a grammar allowing for the combinatorial combination of grammar elements (examples in Figure 5 left in the appendix).
The modiﬁed grammar is given in the appendix.
Figuremiddle shows the results of synthesis using best-ﬁrst search from a policy on test problems sampled the same distribution as the training problems.
Our blended model ﬁnds the highest overall number of correct programs, achieving 5-10% higher accuracy given the same search budget compared to the neural semantics and RNN encoding schemes.
The blended model also yielded superior results on numerous variations of these tasks (increasing number of higher order functions, varying integer ranges, varying search method, etc).
See the appendix for details.
A R ELATED WORK
Synthesizing programs from examples is a classic AI problem which has seen advances from the Programming Languages community (Solar-Lezama, 2008).
Recently, much progress has been made using neural methods to aid search.
Such techniques include enumerative approaches (Balog et al., 2016), translation-based techniques (Devlin et al., 2017), hybrid approaches using sketches such as
Murali et al.
(2017), Nye et al.
and Dong & Lapata (2018), among others.
The recent work of Odena & Sutton (2019) also studies program representation, proposing property signatures.
Work has also been done using graph neural networks to encode the syntax of programs (Allamanis et al., 2018; Brockschmidt et al., 2018) for bug ﬁxing, variable naming, and synthesis.
Recent work has introduced the notion of “execution-guided neural program synthesis” (Ellis et al., 2019; Chen et al., 2018; Zohar & Wolf, 2018).
In this framework, the neural representations used for search are conditioned on the executed program state instead of the program syntax.
These techniques have been shown to solve difﬁcult search problems outside the scope of enumerate or syntax-based neural synthesis alone.
However, such execution-guided approaches have several limitations.
We aim to generalize execution guided synthesis, so that it can be applicable to a wider range of domains, search techniques, and programming language constructs.
Our work is directly inspired by two techniques: abstract interpretation-based synthesis (Wang et al., 2017;
Hu et al., 2019) and neural module networks (Andreas et al., 2016; Johnson et al., 2017).
We employ module networks to implement blended neural execution, which aims to provide a learned execution scheme directly inspired by abstract interpretation.
This approach is also related to other tree-structured encoders (Socher et al., 2011; Dyer et al., 2016).
IO examples 1/
[Figure]
B P ROGRAM S YNTHESIS WITH B LENDED A BSTRACT S EMANTICS
Let X = { ( x i , y i ) } , where ( x i , y i ) are input-output pairs.
Let [[ s ]] blendx i denote the blended abstract semantic representation of s with input x i .
The representation of a state rep ( s, X ) is:
[Equation]
Here, W is a learnable weight matrix, and the representation is averaged across all input-output pairs of X .
Given this state representation, the policy and value function are deﬁned as follows:
[Equation]
Here, MLP is a multi-layer perceptron.
Note the value function outputs a value between 0 and 1 ; this allows for a probabilistic interpretation.
End-to-end training We train our policy π using imitation learning.
Starting from the empty partial program s 0 = HOLE , we generate a sequence of partial programs s 1 , s 2 , · · · by sampling a sequence of expansions a 0 , a 1 , · · · from the grammar G .
Let p =
s T be the completed program.
We obtain speciﬁcations X
= { ( x i , y i ) } by sampling a set of inputs x 1 · · · x n and obtaining outputs using concrete semantics y i =
(cid:74) p (cid:75) cx i .
Thus, from a sequence of expansions a 0 , a 1 , · · · , we can collect a set of triplets { ( X , s i , a i ) } as training data.
This process is repeated to generate the training set D .
We can then perform supervised training, maximizing the log likelihood of the following:
[Equation]
We train the value function by sampling rollouts of partial programs s 0 · · · s T from a fully-trained π , minimizing the error in a Monte-Carlo estimate of the expected reward R (i.e., the probability of success under the policy).
[Equation]
For our error function, we use a logistic loss rather than the more common MSE.
C C OMPARISON TO ABSTRACT INTERPRETATION
How does the learned value function compare to hand-coded abstract interpretation?
Classic abstract interpretation is conservative; it can be thought of as a classiﬁer with perfect recall, but poor precision, only rejecting the partial programs it knows for sure to be unsuitable.
Can our value function also detect these clearly bad partial programs, but ascribe low value to less obviously bad candidates?
To test this, we performed an experiment in the tower-building domain.
We conditioned the model on tasks from our test corpus, and sampled 15 search trajectories from our blended semantics policy for each task.
For each partial program encountered during search, we compute the model’s value judgment, and recorded whether each rollout was successful.
Treating rollout success as a noisy label of partial program quality, and using the value function as a classiﬁer, we plot precision vs recall of the value judgements as we vary the classiﬁcation threshold.
Figure 6 shows our results for this experiment.
As the classiﬁcation threshold is varied, our learned value maintains comparable recall compared to the hand-coded abstraction, while achieving better precision.
For high classiﬁcation thresholds, our model achieves performance comparable to the hand-coded abstract interpretation, and additional precision is gained by lowering the classiﬁcation threshold.
The
RNN value performs worse on this test, achieving lower precision and recall.
D IO PROGRAMMING : S TRING E DITING
In an additional experiment, we examine how our model performs on domains for which executionguided synthesis is possible, but requires extensive changes to the underlying DSL.
For example, in the RobustFill DSL, a function getSubStr(i,j) slices a string from index i to index j .
This function is not executable until both i and j are known.
In order to perform executionguided synthesis, Ellis et al.
(2019) needed to replace getSubString with two separate functions: getSubStrStart i and getSubStrEnd j , where each half can be executed in the REPL.
This process must be performed manually for every language construct which takes multiple arguments.
Here we seek to answer the question: can our model be used to successfully synthesize programs using the language as-is?
To this end, we implement the code-writing policy using the DSL presented in Devlin et al.
(2017) without modiﬁcation (example programs in Figure 5 left).
We do not expect that our approach would outperform the REPL system in Ellis et al.
, but we hope that it could achieve much of the gains.
We additionally compare against another relevant baseline:
RobustFill (Devlin et al., 2017).
In contrast to the original paper, we train the RobustFill model using the same ”unmodiﬁed” version of the DSL as our model, whose syntax has not been modiﬁed to aid with prediction.
At test time, we used a sample-based search procedure, because the branching factor is prohibitively large for breadth-ﬁrst search procedures explored above.
While the blended encoding does not achieve the accuracy of the execution-guided REPL system, it outperforms the other baselines, including the RobustFill model, neural abstract semantics and the
RNN baseline.
E S EMANTICS
Here we fully deﬁne the semantics for concrete, neural, and blended semantics, covering details that were omitted in the main paper.
Concrete semantics In this work, we consider domains where the underlying programming language is functional .
Let λx.
E be a lambda expression of one argument, and let x = v be an assignment of the variable x to the value v .
Lambda application is deﬁned as follows:
[Equation]
That is, we evaluate the function body E , replacing all instances of the function variable x with the value v .
For example, (cid:74) ( λx.x + x )(5) (cid:75) {} = (cid:74) x
+ x (cid:75) { x =5 } = 5ˆ+5 = 10 .
Where ˆ+ denotes the execution semantics of the built-in function + .
The concrete semantics (cid:74) · (cid:75) is deﬁned: [Equation] k
[Equation]
executing built-in f
[Equation]
lambda application
Neural semantics For built-in functions f , we use f nn , a neural module function of k vector inputs, where k is the arity of f .
In our domains, lambda expressions are only used as arguments to higher-order functions.
Therefore, since we will never apply a lambda expression directly under neural semantics, we only require vector representation of lambdas.
However, we still require a mechanism to represent arbitrary lambda expressions built combinatorially from primitive functions.
This representation is constructed in a modular fashion by encoding the body E of the lambda expression.
[[ E ]] nn
[Equation]
=   [[ k ]]
[Equation]
MBED
[Equation]
k ) a constant k is embedded [[ x ]] nn C| =
x = v = E MBED ( v ) embed the value v of the variable x [[ HOLE ]] nn C = h ( C ) a neural placeholder for a hole based on context [[ f ( E 1 · · · )]] nn C = f nn ([[ E 1 ]] nn C · · · ) using neural module [[ λx.
E ]] nn C =
[[ E ]] nn C∪{ x = null } encode the body of a lambda
Note that in representing a lambda expression, we used the context with the assignment x = null.
This is used to account for the fact that, in the time of the lambda function’s deﬁnition (as an argument to a higher-order function), its argument is still unknown under neural execution.
The deﬁnition for blended semantics proceeds in an analogous fashion, with concrete subtrees executed concretely.
F E XPERIMENTAL D ETAILS
All models are trained with the AMSGrad (Reddi et al., 2018) variant of the Adam optimizer with a learning rate of 0.001.
All RNNs are 1-layer and bidirectional GRUs, where the ﬁnal hidden state is used as the output representation.
Unless otherwise stated, holes are encoded by applying the
E MBED function to the context, and then applying a type-speciﬁc neural module to the resulting vector.
F. T OWERS
We employ the tower-building domain and DSL introduced in Ellis et al.
(2020), which consists of the basic commands: PlaceHorizontalBlock , PlaceVerticalBlock , MoveHand(n) ,
ReverseHand() , Embed , Loop and integers n from 1 to 8.
(The higher-order Embed function takes an expression as input, executes it, and then returns the hand to its initial location.)
In order to test the compatibility of our approach with library-learning techniques, we additionally use library functions learned by the DreamCoder system by combining the above functions.
Following Ellis et al.
, the grammar is implemented in continuation-passing style.
Our training data consisted of tower programs randomly sampled from a PCFG generative model (Ellis et al., 2020).
We trained policy networks on 480000 programs.
We trained value functions on 240000 rollouts from the policy.
We perform search for up to 300 seconds per problem.
All neural modules consist of a single linear layer (input dimension 512 ∗ n args and output dimension 512 ) followed by ReLU activation.
Tower images are embedded with a simple CNN-ReLUMaxPool architecture, as in Ellis et al.
Hand-coded abstract interpretation We implemented an abstract domain which tracked, a) the range of possible locations of the “hand” and b) for each horizontal location, the minimum height which must be achieved by the partially constructed tower.
This representation allows us to eliminate invalid partial programs because once a block is dropped, it cannot be removed through any subsequent commands.
F. L IST PROCESSING
Data for this domain was generated by modifying the DeepCoder dataset (Balog et al., 2016).
Specifically, DeepCoder training programs of size 2 (containing 2-3 higher order functions, such as map f (filter g input) or zipwith f
(map g input) (map h input) ) were modiﬁed by changing the lambdas in the program ( f , g , and h in the above examples) from a small set of constant lambdas such as (*2) to depth-3 lambdas sampled from our modiﬁed grammar (see below).
For example: ( λ x.max(x+2,x/2)) .
For each program, 5 example input lists were sampled, each with length 10 and values in the range [-64, 64].
The program was then executed to yield the corresponding outputs.
Programs with output or intermediate values outside of the range [-64, 64] were discarded.
Programs producing the identity function or constant functions were also discarded.
We trained and tested only on functions of type [int] → [int] .
At test time when running search, we similarly reject programs with intermediate values outside of the desired integer range.
All policy networks were trained on 500000 programs.
We perform search for 180 seconds per problem.
All neural modules consist of a single linear layer (input dimension 64 ∗ n args and output dimension 64 ) followed by ReLU activation.
Integers are encoded digit-wise via a GRU.
Lists are encoded via a GRU encoding over the representations of the integers they contain.
Unbound variables within a lambda function are embedded via a learned representation parameterized by the variable name (one vector representing x and one representing y).
When encoding holes within lambda functions, we ignore context, and instead embed holes only as a function of the hole type.
Modiﬁed lambda grammar Below is the grammar used for lambda functions:
L → ( λ x,y [Equation] S) | (
[Equation]
S)
S → I | B
I → I+I | I*I | I/I | min(I,I) | max(I,I)
|
A
B → I > I | or(B,B) | and(B,B) | I%I==
A → x | y | N
N → -2 | -1| 0 | 1 |
Variations on training and testing conditions Many variations on the training and testing conditions achieve similar results to those shown in the main paper (i.e., blended semantics consistently achieves the highest performance).
Several of these variations are shown in Figure 7.
F. S TRING E DITING
For the string editing tasks, we use the DSL from Devlin et al.
We train on randomly sampled programs, sampling I/O pairs and propagating constraints from programs to inputs to ensure that input strings are relevant for the target program (see Devlin et al.
We condition on 4 I/O examples for each program.
We used string editing problems from the SyGuS (Alur et al., 2016) program synthesis competition as our test corpus.
We trained all models on 2 million training programs.
At test time, we sample programs from the model for a maximum timeout of 30 seconds.
Input and output strings are encoded by embedding each character via a 20-dimensional character embedding and concatenating the resulting vectors to form a representation for each string.
Representations of “expressions” e (as deﬁned in the RobustFill DSL) are concatenated together using an “append” module.
Following Ellis et al.
(2019), neural modules consist of a single dense block with 5 layers and a growth rate of 128 (input dimension 256 ∗ n args and output dimension 256 ).
[Figure]