Execution-Guided Neural Program Synthesis

1 INTRODUCTION . Program synthesis is a traditional challenging problem . Such a problem typically takes a specification as the input , and the goal is to generate a program within a target domain-specific language ( DSL ) . One of the most interesting forms of the specifications is input-output examples , and there have been several applications , such as FlashFill ( Gulwani , 2011 ; Gulwani et al. , 2012 ) . Recently , there is an increasing interest of applying neural network approaches to tackle the program synthesis problem . For example , Devlin et al . have demonstrated that using an encoder-decoderstyle neural network , their neural program synthesis algorithm called RobustFill can outperform the performance of the traditional non-neural program synthesis approach by a large margin on the FlashFill task ( Devlin et al. , 2017b ) . Despite their promising performance , we identify several inefficiencies of such encoder-decoder-style neural program synthesis approaches . In particular , such a neural network considers program synthesis as a sequence generation problem ; although some recent work take the syntactical information into consideration during program generation ( Bunel et al. , 2018 ; Rabinovich et al. , 2017 ; Yin & Neubig , 2017 ; Parisotto et al. , 2017 ; Xu et al. , 2017 ) , the semantic information , which is typically well-defined in the target DSL , is not effectively leveraged by existing work . In light of this observation , in this work , we develop simple yet principled techniques that can be combined with any existing encoder-decoder-style neural program synthesizers . The main novel technique is called execution-guided synthesis . The basic idea is to view the program execution as a sequence of manipulations to transform each input state into the corresponding output state . In such a view , executing a partial program can result in intermediate states ; thus , synthesizing the rest of the program can be conditioned on these intermediate states , so that the synthesizer can take the state changes into account in the followup program generation process . Therefore , we can leverage this idea to combine with any existing encoder-decoder-style neural synthesizer , and we observe that it can significantly improve the performance of the underlying synthesizers . In addition , we also propose a simple yet effective technique called synthesizer ensemble , which leverages the semantic information to ensemble multiple neural program synthesizers . In particular , for the input-output program synthesis problem , we can easily verify if a synthesized program satisfies the input-output specification , which allows us to remove invalid predictions from the ensemble during inference time . Albeit its simplicity , to the best of our knowledge , we are not aware of any previous neural program synthesis work applying this approach . We observe that this technique further boosts the performance substantially . We evaluate our techniques on the Karel task ( Bunel et al. , 2018 ; Devlin et al. , 2017a ) , the largest publicly available benchmark for input-output program synthesis , on which the most performant model in the past can achieve only an accuracy of around 77 % ( Bunel et al. , 2018 ) . We observe that our proposed techniques can gain better performance than the previous state-of-the-art results . In particular , by combining both of our techniques , we can achieve an accuracy of 92 % , which is around 15 percentage points better than the state-of-the-art results . This shows that our approach is effective in boosting the performance of algorithms for neural program synthesis from input-output examples . 2 NEURAL PROGRAM SYNTHESIS FROM INPUT-OUPUT EXAMPLES . In this section , we first introduce the input-output program synthesis problem and existing encoderdecoder-style neural program synthesis approaches , then present an overview of our approaches . 2.1 PROBLEM DEFINITION . We follow the literature ( Devlin et al. , 2017b ; Bunel et al. , 2018 ; Chen et al. , 2018 ) to formally define the input-output program synthesis problem below . Problem Definition 1 ( Program emulation ) LetL be the space of all valid programs in the domainspecific language ( DSL ) . Given a set of input-output pairs { ( Ik , Ok ) } Kk=1 ( or { IOK } in short ) , where there exists a program P ∈ L , such that P ( Ik ) = Ok , ∀k ∈ { 1 , ... , K } . Our goal is to compute the output Otest for a new test input Itest , so that Otest = P ( Itest ) . Although the problem definition only requires to compute the output for a test input , a typical method is to synthesize a program P ′ ∈ L such that P ′ is consistent with all input-output pairs { IOK } , and then use P ′ to compute the output . In this case , we say program P ′ emulates the program P corresponding to { IOK } . In particular , in this work , we are mainly interested in the following formulation of the problem . Problem Definition 2 ( Program synthesis ) Let L be the space of all valid programs in the domainspecific language ( DSL ) . Given a training dataset of { IOK } i for i = 1 , ... , N , where N is the size of the training data , compute a synthesizer Γ , so that given a test input-output example set { IOK } test , the synthesizer Γ ( { IOK } test ) = P produces a program P , which emulates the program corresponding to { IOK } test . 2.2 ENCODER-DECODER-STYLE NEURAL PROGRAM SYNTHESIS APPROACHES . There have been many approaches proposed for different neural program synthesis tasks , and most of them follow an encoder-decoder-style neural network architecture ( Bunel et al. , 2018 ; Devlin et al. , 2017b ; Parisotto et al. , 2017 ) . Figure 1 shows a general neural network architecture for input-output program synthesis . First , an encoder converts input-output examples { IOK } into an embedding . For example , RobustFill ( Devlin et al. , 2017b ) deals with the string transformation tasks , thus it uses LSTMs as the encoder . For the Karel task , both inputs and outputs are 2D grids ( see Figure 2 ) ; therefore , Bunel et al . ( 2018 ) employ a CNN as the encoder . Once the IO embeddings are computed , these approaches employ an LSTM decoder to generate the programs conditioned on the embeddings . For program synthesis , one unique property is that the generated program should satisfy the synthax of L. Therefore , a commonly used refinement is to filter syntactically invalid program prefixes during generation ( Parisotto et al. , 2017 ; Devlin et al. , 2017b ; Bunel et al. , 2018 ) . In the above approaches , only the syntax information is leveraged ; the semantics of L is not utilized . In particular , standard supervised training procedure could suffer from program aliasing : for the same input-output examples , there are multiple semantically equivalent programs , but all except the one provided in the training data will be penalized as wrong programs . To mitigate this issue , Bunel et al . propose to train with reinforcement learning , so that it rewards all semantically correct programs once they are fully generated ( Bunel et al. , 2018 ) . In our work , we demonstrate that we can leverage the semantic information in an effective way that provides a better performance . 2.3 AN OVERVIEW OF OUR APPROACHES . In this work , we propose two general and principled techniques that can improve the performance over existing work , which are execution-guided synthesis ( Section 3 ) and synthesizer ensemble ( Section 4 ) . The main idea of our techniques is to better leverage the semantics of the language L during synthesis . Meanwhile , our techniques are compatible with any existing encoder-decoder-style neural program synthesis architecture . We will describe these techniques in detail in the following sections . 3 EXECUTION-GUIDED SYNTHESIS . Existing approaches generate the entire program only based on the input-output examples before execution . However , this is an inefficient use of the semantics of L. For example , when a program consists of a sequence of statements , we can view the output to be a result by continuously executing each statement in the sequence to convert the input state into a sequence of intermediate states . Figure 2 illustrates such an example . From this perspective , instead of generating the whole program at once , we can generate one statement at a time based on the intermediate/output state pairs . However , most interesting programs are not just sequential . In this work , we explore this idea using a general control-flow framework . In particular , given any language L , we extend it with three classical types of control-flow : sequential , branching , and looping . The extended language is call Lext . Then , we develop our above idea based on Lext , called execution-guided synthesis . In the following , to make our discussion concise , we first formalize Lext ( Section 3.1 ) , and then present the idea of execution-guided synthesis ( Section 3.2 ) . 3.1 THE FORMAL SPECIFICATION OF THE EXTENDED LANGUAGE Lext In this work , we assume some additional control-flow syntax on top of L. We define the extended language Lext in Table 1 . In particular , we assume that a code block B can be composed by a sequence of statements S ∈ L or sub code blocks , and each code block can also be an if-statement or a while-statement . We use C to indicate a condition expression in L , and ⊥ to indicate the termination of a program execution . The semantics of Lext is specified in Table 2 . These rules are largely standard following the convention in programming language literature . In particular , each rule ’ s name starts with S- indicating that they are semantics rules ; the suffixes indicate the constructors each rule specifies , ( e.g. , Stmt for statements , Seq for sequences , etc. ) . The judgment 〈B , s〉 → 〈B′ , s′〉 indicates a small-step execution of program B over state s to result in a new program B′ and a new state s′ . The judgments 〈S , s〉 ⇓ s′ and 〈C , s〉 ⇓ b capture the big-step execution in L that statement S evaluates to a new state s′ from s , and condition C evaluates to a boolean value b from s. Following the semantics of Lext , we can formally define a program execution . Definition 1 ( Program execution ) Given a program P ∈ Lext and an input I , the execution is a sequence s0 ... sT , such that ( 1 ) s0 = I ; ( 2 ) B0 = P ; ( 3 ) 〈Bi , si〉 → 〈Bi+1 , si+1〉 for i = 0 , ... , T−1 ; and ( 4 ) BT = ⊥ . The output of the program is O = sT . 3.2 EXECUTION-GUIDED SYNTHESIS ALGORITHM . In Definition 1 , we can observe that the initial and final states are simply two special states provided as the input-output examples of the synthesis problem . Thus , a synthesizer Γ for input-output pairs should also be able to take any state-pairs as inputs . Our execution-guided synthesis algorithm takes advantage of this fact to improve the performance of the synthesizer . In the following , we discuss three cases from the easiest to the hardest to present our approach . Sequential programs . We now consider the simplest case , where the program is in the form of S1 ; ... ; ST , to illustrate the basic idea of execution-guided synthesis algorithm . We present the algorithm in Algorithm 1 . Assuming the input-output examples are { IOK } , we can treat them as K state-pairs { ( ski , sko ) } Kk=1 , where ski = Ik , sko = Ok . The Exec algorithm takes the synthesizer Γ and the input-output pairs { IOK } as its input . It also takes an additional input ∆ , which is the ending token to be synthesized . For the top-level program , ∆ will be the ⊥ token . Later we will see that when synthesizing the sub-program for If- and While-blocks , different ending tokens will be used . The synthesized program is initially empty ( line 3 ) . Then the algorithm iteratively generates one statement S at a time ( line 4-10 and 17 ) , and appends it to the end of P ( line 16 ) . Importantly , if S is not an if-statement or a while-statement , for which we handle separately , the algorithm executes the newly generated statement S to transit ski into sknew ( line 11-14 ) . Therefore , in the subsequent iteration , the synthesizer can start from the new states sknew after executing the partial program P Algorithm 1 Execution-guided synthesis ( sequential case ) 1 : function EXEC ( Γ , { ( ski , sko ) } Kk=1 , ∆ ) 2 : // The main algorithm is called using Exec ( Γ , { IOK } , ⊥ ) 3 : P ← ⊥ 4 : S ← Γ ( { ( ski , sko ) } Kk=1 ) 5 : while S 6= ∆ do 6 : if S = if -token then // If-statement synthesis 7 : S , { ( ski , sko ) } Kk=1 ← ExecIf ( Γ , { ( ski , sko ) } Kk=1 ) 8 : else 9 : if S = while-token then // While-statement synthesis 10 : S , { ( ski , sko ) } Kk=1 ← ExecWhile ( Γ , { ( ski , sko ) } Kk=1 ) 11 : else // Execution of S 12 : 〈S , ski 〉 → 〈⊥ , sknew〉 for k = 1 , ... , K 13 : ski ← sknew for k = 1 , ... , K 14 : end if 15 : end if 16 : P ← P ; S 17 : S ← Γ ( { ( ski , sko ) } Kk=1 ) 18 : end while 19 : return P 20 : end function Algorithm 2 Execution-guided synthesis ( if-statement ) 1 : function EXECIF ( Γ , I ) 2 : C ← Γ ( I ) 3 : It ← { ( si , so ) ∈ I|〈C , si〉 ⇓ true } 4 : If ← { ( si , so ) ∈ I|〈C , si〉 ⇓ false } 5 : Bt ← Exec ( Γ , It , else-token ) 6 : Bf ← Exec ( Γ , If , fi-token ) 7 : I ′t ← { ( snew , so ) | ( si , so ) ∈ It ∧ 〈Bt , si〉 ⇓ snew } 8 : I ′f ← { ( snew , so ) | ( si , so ) ∈ If ∧ 〈Bf , si〉 ⇓ snew } 9 : I ← I ′t ∪ I ′f 10 : S ← if C then Bt else Bf fi 11 : return S , I 12 : end function generated so far . In doing so , the synthesizer can see all intermediate states to better adjust the followup synthesis strategies to improve the overall synthesis performance . Branching programs . Dealing with if-statements is slightly more complicated than sequential programs , since in an if-statement , not all statements will be executed on all inputs . Following the execution in Algorithm 1 naively , we have to use Γ to synthesize the entire if-statement before being able to execute the partially generated program to derive intermediate states . Therefore , in Algorithm 2 , we extend the above idea to handle if-statements . When the next predicted token is an if -token , our execution-guided synthesizer first predicts the condition of the if-statement C ( line 2 ) . Then , we evaluate C over all state-pairs . Based on the evaluation results , we can divide the IO pairs into two sets It and If ( line 3-4 ) , so that all states in the former meet the branching condition to go to the true branch , and all states in the latter go to the false branch . Therefore , in the followup synthesis , we do not need to consider If ( or It ) when synthesizing the true branch ( or the false branch ) ( line 5-6 ) . Note that in line 5-6 , synthesizing both true-branch and false-branch employ execution-guided synthesis algorithm to leverage intermediate states , and different ending tokens ( i.e. , else and fi ) are supplied respectively . Once we have done the synthesis of both branches , we can execute the generated branches to get the new states I ( line 7-9 ) , and return the newly generated if-statement and updated states to the caller of this algorithm . In Algorithm 2 , we use 〈B , s〉 ⇓ s′ to indicate a big-step execution of code block B over state s to get s′ . In particular , this means that 〈B , s〉 → 〈B1 , s1〉 → ... → 〈⊥ , s′〉 . Looping programs . The remaining problem is to handle while-statements . Due to the rule S-While ( see Table 2 ) , a while-statement while C do B end ( 1 ) is equivalent to if C then ( B ; while C do B end ) else ⊥ fi ( 2 ) Therefore , we can employ a procedure similar to Algorithm 2 once a while-token is predicted . However , there are two differences . First , in ( 2 ) , the false-branch is empty , thus we do not need to deal with the false-branch . Second , although the true-branch is B ; while C do B end , once we have generated B , we do not need to generate the rest of the true-branch , since both C and B have been generated . The detailed algorithm can be found in Appendix B . Remarks . The final algorithm is called by Exec ( Γ , { IOK } , ⊥ ) . Note that our execution-guided synthesis algorithm can be applied to any neural synthesizer Γ , and we can train the synthesizer Γ using any supervised or reinforcement learning algorithms that have been proposed before ( Devlin et al. , 2017b ; Bunel et al. , 2018 ) . In our evaluation , we demonstrate that our execution-guided synthesis technique helps boost the performance of both supervised and reinforcement learning algorithms proposed in existing work ( Bunel et al. , 2018 ) .

Summary: This paper proposes guiding program synthesis with information from partial/incomplete program execution. The idea is that by executing partial programs, synthesizers can obtain the information of the state the (partial) program ended in and can, therefore, condition the next step on that (intermediate) state. The paper also mentions ensembling synthesizers to achieve a higher score, and by doing that it outperforms the current state-of-the-art on the Karel dataset program synthesis task.