Task: Language Identification
Input (Spanish, English): I'd like to thank the authors for their detailed response and clarifications.  This work proposes new training scheme for online sparse dictionary learning. The model assumes a non-stationary flow of the incoming data. The goal (and the challenge) is to learn a model in an online manner in a way that is capable of  adjusting to the new incoming data without forgetting how to represent previously seen data. The proposed approach deals with this problem by incorporating a mechanism for adding or deleting atoms in the dictionary. This procedure is inspired by the adult neurogenesis phenomenon in the dentate gyrus of the hippocampus.   The paper has two main innovations over the baseline approach (Mairal et al): (i) “neuronal birth” which represents an adaptive way of increasing the number of atoms in the dictionary (ii) "neuronal death", which corresponds to removing “useless” dictionary atoms.  Neural death is implemented by including an group-sparsity regularization to the dictionary atoms themselves (the group corresponds to a column of the dictionary). This promotes to shrink to zero atoms that are not very useful, keeping controlled the increase of the dictionary size.  I believe that the strong side of the paper is its connections with the adult neurogenesis phenomenon, which is, in my opinion a very nice feature. The paper is very well written and easy to follow.  On the other hand, the overall technique is not very novel. Although not exactly equivalent, similar ideas have been explored. While the neural death is implemente elegantly with a sparsity-promoting regularization term, the neural birth is performed by relying on heuristics that measure how well the dictionary can represent new incoming data. Which depending on the "level" of non-stationarity in the incoming data (or presence of outliers) could be difficult to set. Still, having adaptive dictionary size is very interesting.  The authors could also cite some references in model selection literature. In particular, some ideas such as MDL have been used for automatically selecting the dictionary size (I believe this work does not address the online setting, but still its a relevant reference to have). For instance,  Ramirez, Ignacio, and Guillermo Sapiro. "An MDL framework for sparse coding and dictionary learning." IEEE Transactions on Signal Processing 60.6 (2012): 2913-2927.
Output (English): en