In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
--------
Question: This paper is very interesting, I like the idea, and to make it a work publishable in vennue reaching a wider international audience it is necessary to validate the proposed ideas through extensive experimentation with actual data and users, please go ahead.

Answer: en


Question: Paper Summary:  Authors investigate identity re-parametrization in the linear and the non linear case.   Detailed comments:  — Linear Residual Network:  The paper shows that for a linear residual network any critical point is a global optimum. This problem is non convex it is interesting that this simple re-parametrization leads to such a result.    — Non linear Residual Network:  Authors propose a construction that maps the points to their labels via a resnet , using an initial random projection, followed by a residual block that clusters the data based on their label, and a last layer that maps the clusters to the label.   1- In Eq 3.4  seems the dimensions are not matching q_j in R^k and e_j in R^r. please clarify   2- The construction seems fine, but what is special about the resnet here in this construction? One can do a similar construction if we did not have the identity? can you discuss this point? In the linear case it is clear from a spectral point of view how the identity is helping the optimization. Please provide some intuition.    3-   Existence of a network in the residual  class that overfits does it give us any intuition on why residual network outperform other architectures? What does an existence result of such a network tell us about its representation power ?  A simple linear model under the assumption that points can not be too close can overfit the data, and get fast convergence rate (see for instance tsybakov noise condition).  4- What does the construction tell us about the number of layers?   5- clustering the activation independently from the label, is an old way to pretrain the network. One could use those centroids as weights for the next layer (this is also related to Nystrom approximation see for instance

Answer: en


Question: This paper describes how to approximate the FastText approach such that its memory footprint is reduced by several orders of magnitude, while preserving its classification accuracy. The original FastText approach was based on a linear classifier on top of bag-of-words embeddings. This type of method is extremely fast to train and test, but the model size can be quite large.  This paper focuses on approximating the original approach with lossy compression techniques. Namely, the embeddings and classifier matrices A and B are compressed with Product Quantization, and an aggressive dictionary pruning is carried out. Experiments on various datasets (either with small or large number of classes) are conducted to tune the parameters and demonstrate the effectiveness of the approach. With a negligible loss in classification accuracy, an important reduction in term of model size (memory footprint) can be achieved, in the order of 100~1000 folds compared to the original size.  The paper is well written overall. The goal is clearly defined and well carried out, as well as the experiments. Different options for compressing the model data are evaluated and compared (e.g. PQ vs LSH), which is also interesting. Nevertheless the paper does not propose by itself any novel idea for text classification. It just focuses on adapting existing lossy compression techniques, which is not necessarily a problem. Specifically, it introduces:   - a straightforward variant of PQ for unnormalized vectors,   - dictionary pruning is cast as a set covering problem (which is NP-hard), but a greedy approach is shown to yield excellent results nonetheless,   - hashing tricks and bloom filter are simply borrowed from previous papers.  These techniques are quite generic and could as well be used in other works.    Here are some minor problems with the paper:    - it is not made clear how the full model size is computed. What is exactly in the model? Which proportion of the full size do the A and B matrices, the dictionary, and the rest, account for? It is hard to follow where is the size bottleneck, which also seems to depend on the target application (i.e. small or large number of test classes). It would have been nice to provide a formula to calculate the total model size as a function of all parameters (k,b for PQ and K for dictionary, number of classes).      - some parts lack clarity. For instance, the greedy approach to prune the dictionary is exposed in less than 4 lines (top of page 5), though it is far from being straightforward. Likewise, it is not clear why the binary search used for the hashing trick would introduce an overhead of a few hundreds of KB.     Overall this looks like a solid work, but with potentially limited impact research-wise.

Answer:
en