Neural Program Synthesis with Query

1 INTRODUCTION . Program synthesis is the task of automatically finding a program that satisfies the user intent expressed in the form of some specifications like input-output examples ( Gulwani et al. , 2017 ) . Recently , there has been an increasing interest in tackling it using neural networks in various domains , including string manipulation ( Gulwani , 2011 ; Gulwani et al. , 2012 ; Devlin et al. , 2017b ) , list processing ( Balog et al. , 2017 ; Zohar & Wolf , 2018 ) and graphic applications ( Ellis et al. , 2018 ; 2019 ) . Despite their promising performance , most of their success relies on the well-designed input-output examples , without which the performance of the program synthesis model drops heavily . Traditionally , the input-output examples are selected elaborately . For example , in Karel ( Devlin et al. , 2017b ; Bunel et al. , 2018 ) , the given input-output examples are required to have a high branch coverage ratio on the test program ; In list processing , the given input-output examples are guided by constraint propagation ( Balog et al. , 2017 ) to ensure their effectiveness on the unknown program . However , providing such input-output examples is unrealistic because it requires the users to be experienced programmers to describe the underlying program with several input-output examples . Worse , the users must be familiar with the distribution of the training dataset to avoid input-output examples that are out-of-distribution ( Shin et al. , 2019 ) . Some researchers have tried to solve this problem by search or random selection to find counter examples iteratively ( Jha et al. , 2010 ; Laich et al. , 2020 ; Hajipour et al. , 2020 ) . However , they do not model the similarity between the input-output examples and programs accurately , and thus the quality of the examples can not be guaranteed . In summary , how to generate informative input-output examples without expert experience is still an important problem that remains a challenge . In this paper , we propose a query-based framework that can tackle the above-mentioned problems efficiently . Inspired by the interactive program synthesis , the query-based framework trains a query neural network to generate informative input-output examples by querying the underlying program iteratively . Specifically , we firstly train a query network with programs and then use it to generate informative input-output examples . After that , we use these generated examples to train the program synthesis network . This framework has three advantages : ( 1 ) There is no need for well-designed input-output examples in both training and testing , which leads to a good generalization . ( 2 ) The query network works in a generative manner , which can essentially reduce the computation cost Predicted programIOs Generate SynthesizeInteract Embed Query network Synthesis networkUser intent ( program ) Figure 1 : The query-based framework . The query network generates informative input-output examples by interacting with the user , and then the generated examples are sent to the synthesis network to synthesize the underlying program . while facing problems with large input-output space . ( 3 ) The query network serves as a plug-andplay module with high scalability , for it is separate from the program synthesis process entirely . To train the query network , we model the similarity between input-output examples and programs by introducing the functional space ( F-space ) and defining the distance between programs in it . Each program can be projected into a vector in F-space . The distance between two vectors indicates the functional difference between these two programs , so if two programs are projected into the same vector in the F-space , they are supposed to be functionally equivalent . Each input-output example corresponds to a set of programs that satisfy it , and thus it can be represented by a set of vectors in F-space . Furthermore , we project this set of vectors to a distribution ( Sun & Nielsen , 2019 ) to make the representation continuous and differentiable , and thus can be optimized by the back-propagation algorithm . With the concept of information theory , we train the query network to generate queried input-output examples that maximize the probability of the corresponding program and minimize the probability of the others , resulting in the maximization of the mutual information between the queries and the corresponding programs . In this way , the query network can generate informative queried input-output examples efficiently . We evaluate our method on the Karel task and the list processing task . Without utilizing the welldesigned input-output examples both in training and testing , we achieve and even outperform the state-of-the-art performance on the Karel task and list processing task , which shows the effectiveness and generalization of our query-based framework . 2 OVERVIEW . In this section , we give the statement of the query problem and then formalize it . 2.1 PROBLEM STATEMENT . To give an intuition of the query problem , consider an oracle that contains some kind of symbolic rules ( e.g.programs ) and our goal is to discover the symbolic rules inside this oracle . To do this , the traditional program synthesis assumes that the oracle can provide some informative signals automatically , based on which a well-designed synthesizer can find the rules . However , this is a strong assumption on the oracle and can not cover every situation . In this work , we consider a more general problem called the query problem , that the informative signals are gained actively by querying the oracle , and the only requirement for the oracle is that it can respond to all kinds of queries ( including meaningless ones ) . The crux of this query problem is , how to generate queries that can help us to identify the underlying rules in the oracle efficiently ? For the sake of illustration , consider the program synthesis task , the oracle corresponds to the underlying program p∗ that satisfies the user intent , and the ( query , response ) pairs correspond to the input-output examples JeK = { ( xk , yk ) } K . To measure different queries , we borrow the terminology ” acquisition function ” from Bayesian optimization . The acquisition function scores all possible queries and the query algorithm chooses the one with the highest score as the next query . We set the acquisition function to be the mutual information between the queries and the oracle I ( JeK ; p∗ ) , maximizing which brings more information gain . Specifically , given input-output examples queried so far JeKt−1 = { ( x1 , y1 ) , ( x2 , y2 ) , · · · ( xt−1 , yt−1 ) } , we train the query network to pick the query xt which obtains the response yt and can maximize I ( JeK ∪ { ( xt , yt ) } ; p∗ ) . The mutual information is hard to calculate analytically . Thus , to approximate the I ( JeK ∪ { ( xt , yt ) } ; p∗ ) , we adopt the InfoNCE ( van den Oord et al. , 2018 ) and propose a novel way to model the relationship between programs and input-output examples called functional space ( F-space ) . Before dive into the specific method , we will give a problem formulation to make our problem description more accurate . 2.2 PROBLEM FORMULATION . To lay a theoretical foundation of our query method , we give out the problem formulation as follows . First , we define the functional equivalence , which is consistent with the definition of equivalence of functions in mathematics . Definition 2.1 ( Functional equivalence ) . Let I be the input example domain containing all valid input examples , O be the output example domain containing all possible output examples , and P be the program domain containing all valid programs under the domain specific language ( DSL ) , each program p ∈ P can be seen as a function p : I → O . For two programs pi ∈ P and pj ∈ P , pi and pj are functional equivalent if and only if ∀x ∈ I , pi ( x ) = pj ( x ) . Using the concept of functional equivalence , we can formulate the program synthesis task : Definition 2.2 ( Program synthesis ) . Suppose there is an underlying program p ∈ P and K inputoutput examples JeK = { ( xk , yk ) | ( xk , yk ) ∈ I × O } K generated by it , the program synthesis task aims to find a program p̂ which is functional equivalent to p using JeK . Following the definitions above , we can define the task of query . Definition 2.3 ( Query ) . The process of query is a map fq . Given a program p ∈ P and a set of history K input-output examples JeK = { ( xk , yk ) | ( xk , yk ) ∈ I × O } K , the process of query generates a query xk+1 = fq ( JeK ) ∈ I , and then get a response yk+1 = p ( xk+1 ) ∈ O . The query and response are added to history input-output examples : JeK ← JeK ∪ { ( xk+1 , yk+1 ) } and this process repeats . The query process aims to distinguish the underlying program with as few queries as possible , which is consistent with the acquisition function that each query should contain as much information as possible . 3 METHODS . The query network and the synthesis network are trained separately . The query network is trained to find the query contains that is most informative , and after that , the synthesis network is trained to predict the corresponding program using the queried input-output examples . In the following , first , we introduce the F-space to model the similarity between the input-output examples and programs , and then , we illustrate how to train the query network with F-space . 3.1 F-SPACE . To model the query process with a neural network , we propose the functional space ( F-space ) . Definition 3.1 ( F-space and functional distance ) . F-space is a |I| dimensional space which consists of all valid programs that can be implemented by program domain P. Each program is represented by |I| different output examples v = ( y1 , y2 , . . . , y|I| ) . The distance in F-space can be measured by the number of different output examples : d ( vi , vj ) = |diff ( vi , vj ) | . Intuitively , F-space ( P , d ) measures the functional differences between programs . Each program can be represented by a vector in F-space , and if different programs are represented by the same vector v = ( y1 , y2 , . . . , y|I| ) in F-space , it indicates that these two programs get the same result for all inputs , and they are considered functional equivalent which is consistent with Definition 2.1 . In practice , a space with dimension |I| is too large to compute . To solve this problem , we utilize the sparsity of F-space and learn an approximate space with dimension reduction by neural networks . Regarding input-output examples , representing them by vectors is not appropriate . Consider a set of input-output examples JeK = { ( xk , yk ) } K , we can not find a vector that represents it when K < |I| because there are more than one programs which satisfies these K input-output examples . For example , when there is only one input-output example JeK = { ( 2 , 4 ) } , we can not distinguish the underlying program y = 2x from y = x2 . Thus , when K < |I| , the input-output examples JeK = { ( xk , yk ) } K should be modeled as a set of vectors in F-space , projected by the set of programs that satisfy them . We summarize the properties of the representation of input-output examples in F-space as follows : • Each set of input-output examples JeK = { ( xk , yk ) } K should be represented by a set of F-space vectors JrK = { vn } N . • For two sets of input-output examples JeKi and JeKj , if JeKi ⊆ JeKj , then their F-space representations JrKi and JrKj have the relationship JrKi ⊇ JrKj . • For two sets of input-output examples JeKi and JeKj , suppose JeK′ = JeKi ∩ JeKj , then in F-space , their corresponding representations have the relationship JrK′ = JrKi ∪ JrKj • Following the property above , if JeK′ = JeKi ∪ JeKj , then in F-space , their corresponding representations have the relationship JrK′ = JrKi ∩ JrKj Representing input-output examples by sets satisfies all these properties already . However , to optimize the representation using neural networks , we need to make it continuous and differentiable . Thus , we model the representation of JeK in F-space as a Normal distribution ( Ren & Leskovec , 2020 ; Sun & Nielsen , 2019 ) , and the probability of the distribution indicates the possibility that it is the underlying program to be synthesized given input-output examples JeK . We illustrate the projection , the subset relationship , and the union/intersection operation of distributions in Figure 2 . Under this representation , the query process becomes the process of reducing the uncertainty of the distribution . As described in Section 2 , the query process aims to distinguish the underlying program with as few queries as possible . In F-space , the equivalent description is , the training of the query network aims to find F-space using as few queries as possible . To train the query network , we define several neural operators for the neural network training as follows . Program projection . To project a program p to a vector v , we traditionally use a sequence model as our encoder : v = Encoderp ( p ) . ( 1 ) Note that the program synthesis model differs largely on different datasets , so we choose to vary our program encoder according to the specific program synthesis models on different datasets . Mostly , our program encoder is the reverse of the program synthesis decoder . Input-output example projection . Given one input-output example JeK = { ( x , y ) } , we can project it into a Normal distribution using the same architecture of the corresponding program synthesis model , except that we add a MLP to output the two parameters of the Normal distribution : µ and log ( σ2 ) . [ µ , log ( σ2 ) ] = MLPe ( Encodere ( JeK ) ) , ( 2 ) where MLP means a multi-layer perceptron . Using this projection , we can model JeK as N ( µ , log ( σ2 ) ) in F-space . Input-output examples intersection . Given K input-output examples JeK = { ( xk , yk ) } K , each example { ( xk , yk ) } is represented by a Normal distribution N ( µk , σk ) using the projector above . The purpose of the intersection is to aggregate these Normal distributions into a new one , which represents JeK in F-space . Under the assumption of independence , the probability of the intersection distribution should be : Printersection = K∏ i=1 Pri . ( 3 ) Fortunately , the product of independent Normal distributions is still a Normal distribution , which means that we can represent it in [ µ′ , log ( σ′2 ) ] . In practice , we utilize the attention mechanism with another MLP to let the neural network learn the new Normal distribution ( Ren & Leskovec , 2020 ) : [ µ′ , log ( σ′2 ) ] = K∑ i=1 wi [ µi , log ( σ 2 i ) ] , ( 4 ) wi = exp ( MLPattention ( [ µi , log ( σ 2 i ) ] ) ) ∑K j=1 exp ( MLPattention ( µj , log ( σ 2 j ) ) ) . ( 5 ) This formulation not only keeps the form of distributions closed but also approximates the mode of the effective support of Normal distributions , which satisfies our requirement on the intersection of distribution ( Sun & Nielsen , 2019 ) . Inverse projection to query . Finally , we need to generate the query from the representation of input-output examples in F-space . Using the projection and the intersection operation mentioned above , we can project the K input-output examples to the representation [ µ , log ( σ2 ) ] . To generate query xnew from this representation , we introduce a decoder which has a similar architecture to Encodere but reversed : xnew = Decoder ( [ µ , log ( σ 2 ) ] ) . ( 6 ) Next , we will introduce how to train the neural network .

Summary: This paper considers an active diagnostic setup for program synthesis as follows: There is an oracle function/program p* belonging to a space of functions, and at inference time, one would like to uncover p*. To do so, a querying algorithm iteratively/adaptively query the oracle with inputs x1, x2, ... xk, receiving outcomes y1, y2, ... yk in return. A synthesizer (or any inverse model capable mapping observations to hypothesis, doesn't really matter) then proposes a guess p' based on the collected query-observation pairs (called examples) {(x1,y1)...(xk,yk)}. One "wins the game" if p* and p` behave the same way (functional equivalent) on all inputs, using as few queries (i.e. small k) as possible.  To do this, the paper introduce a learned F-space, which both functions f and sets of examples e can map into. While functions map (using a learned embedding) to single points in F-space (each point in F-space represent a unique function), examples map (using yet another learned embedding) to distributions in F-space (each example set represent a _distribution of_ functions that are consistent with it). The querying network is trained to generate the next observation x conditioned on the existing observations {(x1,y1)...(xk,yk)} in such a way that maximally reduce the uncertainty of the F-space once the new example, (x, p*(x)) is augmented to the existing observations.  Results are convincing to me.