Task: Language Identification
Input (Spanish, English): This paper describes a correction technique to combine updates from multiple SGD to make it statistically equivalent to sequential technique. Comments 1) The proposed method is novel and interesting to allow update to be corrected even when the update is delayed. 2) The proposed theory can only be applied to square loss setting (with linear update rule), making it somewhat limited. This paper would be much more interesting to ICLR community, if the technique is applicable to general objective function and settings of deep neural networks. 3) The resulting technique requires book-keeping of a dimensional reduced combiner matrix, which causes more computation in terms of complexity. The authors argue that the overhead can be canceled with SIMD support for symbolic update. However, the normal update of SGD might also benefit from SIMD, especially when the dataset is dense.  Overall, even though the practical value of this work is limited by 2) and 3), the technique(specifically the correction rule) proposed in the paper could be of interest to people scaling up learning. I would encourage the author to extend the method to the cases of non-linear objective function which could make it more interesting to the ICLR community
Output (English): en