Task: Language Identification
Input (Spanish, English): Paper Summary:  Authors investigate identity re-parametrization in the linear and the non linear case.   Detailed comments:  — Linear Residual Network:  The paper shows that for a linear residual network any critical point is a global optimum. This problem is non convex it is interesting that this simple re-parametrization leads to such a result.    — Non linear Residual Network:  Authors propose a construction that maps the points to their labels via a resnet , using an initial random projection, followed by a residual block that clusters the data based on their label, and a last layer that maps the clusters to the label.   1- In Eq 3.4  seems the dimensions are not matching q_j in R^k and e_j in R^r. please clarify   2- The construction seems fine, but what is special about the resnet here in this construction? One can do a similar construction if we did not have the identity? can you discuss this point? In the linear case it is clear from a spectral point of view how the identity is helping the optimization. Please provide some intuition.    3-   Existence of a network in the residual  class that overfits does it give us any intuition on why residual network outperform other architectures? What does an existence result of such a network tell us about its representation power ?  A simple linear model under the assumption that points can not be too close can overfit the data, and get fast convergence rate (see for instance tsybakov noise condition).  4- What does the construction tell us about the number of layers?   5- clustering the activation independently from the label, is an old way to pretrain the network. One could use those centroids as weights for the next layer (this is also related to Nystrom approximation see for instance
Output (English): en