Task: Language Identification
Input (Spanish, English): The authors propose a method to investigate the predictiveness of intermediate layer activations. To do so, they propose training linear classifiers and evaluate the error on the test set.  The paper is well motivated and aims to shed some light onto the progress of model training and hopes to provide insights into deep learning architecture design.  The two main reasons for why the authors decided to use linear probes seem to be: - convexity - The last layer in the network is (usually) linear  In the second to last paragraph of page 4 the authors point out that it could happen that the intermediate features are useless for a linear classifier. This is correct and what I consider the main flaw of the paper. I am missing any motivation as to the usefulness of the suggested analysis to architecture design. In fact, the example with the skip connection (Figure 8) seems to suggest that skip connections shouldn't be used. Doesn't that contradict the recent successes of ResNet?  While the results are interesting, they aren't particularly surprising and I am failing to see direct applicability to understanding deep models as the authors suggest.
Output (English): en