Task: Language Identification
Input (Spanish, English): This paper develops a differentiable interpreter for the Forth programming language. This enables writing a program "sketch" (a program with parts left out), with a hole to be filled in based upon learning from input-output examples. The main technical development is to start with an abstract machine for the Forth language, and then to make all of the operations differentiable. The technique for making operations differentiable is analogous to what is done in models like Neural Turing Machine and Stack RNN. Special syntax is developed for specifying holes, which gives the pattern about what data should be read when filling in the hole, which data should be written, and what the rough structure of the model that fills the hole should be. Motivation for why one should want to do this is that it enables composing program sketches with other differentiable models like standard neural networks, but the experiments focus on sorting and addition tasks with relatively small degrees of freedom for how to fill in the holes.  Experimentally, result show that sorting and addition can be learned given strong sketches.  The aim of this paper is very ambitious: convert a full programming language to be differentiable, and I admire this ambition. The idea is provocative and I think will inspire people in the ICLR community.  The main weakness is that the experiments are somewhat trivial and there are no baselines. I believe that simply enumerating possible values to fill in the holes would work better, and if that is possible, then it's not clear to me what is practically gained from this formulation. (The authors argue that the point is to compose differentiable Forth sketches with neural networks sitting below, but if the holes can be filled by brute force, then could the underlying neural network not be separately trained to maximize the probability assigned to any filling of the hole that produces the correct input-output behavior?)  Related, one thing that is missing, in my opinion, is a more nuanced outlook of where the authors believe this work is going. Based on the small scale of the experiments and from reading other related papers in the area, I sense that it is hard to scale up differentiable forth to large real-world problems. It would be nice to have more discussion about this, and perhaps even an experiment that demonstrates a failure case. Is there a problem that is somewhat more complex than the ones that appear in the paper where the approach does not work? What has been tried to make it work? What are the failure modes? What are the challenges that the authors believe need to be overcome to make this work.  Overall, I think this paper deserves consideration for being provocative. However, I'm hesitant to strongly recommend acceptance because the experiments are weak.
Output (English): en