Task: Language Identification
Input (Spanish, English): EDIT: Updated score. See additional comment.  I quite like the main idea of the paper, which is based on the observation in Sec. 3.0 - that the authors find many predictable patterns in the independent evolution of weights during neural network training. It is very encouraging that a simple neural network can be used to speed up training by directly predicting weights.  However the technical quality of the current paper leaves much to be desired, and I encourage the authors to do more rigorous analysis of the approach. Here are some concrete suggestions:  - The findings in Section 3.0 which motivate the approach, should be clearly presented in the paper. Presently they are stated as anecdotes.  - A central issue with the paper is that the training of the Introspection network I is completely glossed over. How well did the training work, in terms of training, validation/test losses? How well does it need to work in order to be useful for speeding up training? These are important questions for anyone interested in this approach.  - An additional important issue is that of baselines. Would a simple linear/quadratic model also work instead of a neural network? What about a simple heuristic rule to increase/decrease weights? I think it's important to compare to such baselines to understand the complexity of the weight evolution learned by the neural network.  - I do not think that default tensorflow example hyperparameters should be used, as mentioned by authors on OpenReview. There is no scientific basis for using them. Instead, first hyperparameters which produce good results in a reasonable time should be selected as the baseline, and then added the benefit of the introspection network to speed up training (and reaching a similar result) should be shown.  - The authors state in the discussion on OpenReview that they also tried RNNs as the introspection network but it didn't work with small state size. What does "didn't work" mean in this context? Did it underfit? I find it hard to imagine that a large state size would be required for this task. Even if it is, that doesn't rule out evaluation due to memory issues because the RNN can be run on the weights in 'mini-batch' mode. In general, I think other baselines are more important than RNN.  - A question about jump points:  The I is trained on SGD trajectories. While using I to speed up training at several jump points, if the input weights cross previous jump points, then I gets input data from a weight evolution which is not from SGD (it has been altered by I). This seems problematic but doesn't seem to affect your experiments. I feel that this again highlights the importance of the baselines. Perhaps I is doing something extremely simple that is not affected by this issue.  Since the main idea is very interesting, I will be happy to update my score if the above concerns are addressed.
Output (English): en