In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
--------
Question: This paper is an extension of Lenc&Vedaldi15 paper, showing CNN representations at FC7 layer are to certain extent equivariant to various classes of transformations and that training with a certain group of transformation makes the representations more equivalent.  Authors performed a large amount of experiments, training over 30 networks with different forms of jitter, which is quite impressive. However it is rather difficult to find a main message of this work. Yes, authors measured the properties on a different layer than the Lenc&Vedaldi15, however it is hard to find some novel insights other than the known fact that jitter helps to achieve invariance. The evaluation seems to be mostly correct, however the paper does not seem to be solving the task advertised in its title really well.  Major issues are in the experiments with the representation distances: * The selection of only FC7 is a bit controversial - it is followed only by a single classification layer (FC8) to the common output - class likelyhoods. Because the FC8 is just a linear projections, what the equivalence map does is just to re-project the FC8 weights of the attached network to the weights of the original network. Probably performing similar experiments but on more layers may be more useful (as the networks are already trained). * The experiment with representation distance is missing what is the classification error on the testing dataset. This would answer whether the representations are actually compatible up to linear transformation at all... * It is not clear for the experiment with K-NN whether this is measured per each test set example? After training the equivalence map? More clear would be to show that networks trained on similar group of jitter transformations are more compatible on the target task. * The proposed method does not seem to improve equivariance consistently on all tasks. Especially with \lambda_1 and \lambda_2 having such small values, the loss is basically equal to simple data jitter as it just adds up the loss of the original and transformed image. Maybe the issue is in the selection of the FC7 layer?  In general, this paper shows some interesting results on the FC7 equivariance, but it does not seem to be drawing many interesting new observations out of these experiments. Due to some issues with the equivalence experiments and the finetuning of equivariance, I would not recommend acceptance of this manuscript. However, refining the experiments on already trained networks and restructuring this manuscript into more investigative work may lead to interesting contribution to the field.  There are also few minor issues: * It is not experimentally verified that the new criterion for equivariance mapping helps to gain better results. * The angles on page 1 and 5 are missing units (degrees?). * On page three, "In practice, it is difficult... ", it is not M_g which is maximised/minimised, but the loss over the M_g * Page 4, footnote 2 - if you are just halving the activations, it is hard to call it a dropout as this constant factor can be passed to the following/preceding weights * Is the network for RVL-CDIP the same architecture as Alexnet? * On page 7, Figure 3a+3b - in my opinion, turning the diagonal elements to white is really misleading, and probably even incorrect, as the distance between the same representations should be zero (which is also a way how to verify that the experiments are performed correctly).

Answer: en


Question: Important task (attention models), interesting distillation application, well-written paper. The authors have been responsive in updating the paper, adding new experiments, and being balanced in presenting their findings. I support accepting this paper.

Answer: en


Question: This paper presents an approach to learn to generate programs. Instead of directly trying to generate the program, the authors propose to train a neural net to estimate a fix set of attributes, which then condition a search procedure. This is an interesting approach, which make sense, as building a generative model of programs is a very complex task.  Faster computation times are shown in the experimental section with respect to baselines including DFS, Enumeration, etc. in a setup with very small programs of length up to 5 instructions have to be found.  It is not clear to me how the proposed approach scales to larger programs, where perhaps many attributes will be on. Is there still an advantage?  The authors use as metric the time to find a single program, whose execution will result in the set of 5 input-output pairs given as input. However, as mentioned in the paper, one is not after a generic program but after the best program, or a rank list of all programs (or top-k programs) that result in a correct execution. Could the authors show experiments in this setting? would still be useful to have the proposed approach? what would the challenges be in this more realistic scenario?  In the second experiment the authors show results where the length of the program at training time is different than the length at test time. However, the results are shown when only 20% of the programs are finished. Could you show results for finding all programs?   The paper is missing an analysis of the results. What type of programs are difficult? how often is the NNet wrong? how does this affect speed? what are the failure modes of the proposed method?  The authors proposed to have a fix-length representation of the each input-output pair, and then use average pooling to get the final representation. However, why would average pooling make sense here? would it make more sense to combine the predictions at the decoder, not the encoder?  Learning from only 5 executions seems very difficult to me. For programs so small it might be ok, but going to more difficult and longer programs this setting does not seem reasonable.   In summary an interesting paper. This paper tackles a problem that is outside my area of expertise so I might have miss something important.

Answer:
en