Task: Language Identification
Input (Spanish, English): This paper presents an approach to learn to generate programs. Instead of directly trying to generate the program, the authors propose to train a neural net to estimate a fix set of attributes, which then condition a search procedure. This is an interesting approach, which make sense, as building a generative model of programs is a very complex task.  Faster computation times are shown in the experimental section with respect to baselines including DFS, Enumeration, etc. in a setup with very small programs of length up to 5 instructions have to be found.  It is not clear to me how the proposed approach scales to larger programs, where perhaps many attributes will be on. Is there still an advantage?  The authors use as metric the time to find a single program, whose execution will result in the set of 5 input-output pairs given as input. However, as mentioned in the paper, one is not after a generic program but after the best program, or a rank list of all programs (or top-k programs) that result in a correct execution. Could the authors show experiments in this setting? would still be useful to have the proposed approach? what would the challenges be in this more realistic scenario?  In the second experiment the authors show results where the length of the program at training time is different than the length at test time. However, the results are shown when only 20% of the programs are finished. Could you show results for finding all programs?   The paper is missing an analysis of the results. What type of programs are difficult? how often is the NNet wrong? how does this affect speed? what are the failure modes of the proposed method?  The authors proposed to have a fix-length representation of the each input-output pair, and then use average pooling to get the final representation. However, why would average pooling make sense here? would it make more sense to combine the predictions at the decoder, not the encoder?  Learning from only 5 executions seems very difficult to me. For programs so small it might be ok, but going to more difficult and longer programs this setting does not seem reasonable.   In summary an interesting paper. This paper tackles a problem that is outside my area of expertise so I might have miss something important.
Output (English): en