Task: Language Identification
Input (Spanish, English): Taking into account the loss in the binarization step through a proximal Newton algorithm is a nice idea. This is at least one approach to bringing in the missing loss in the binarization step, which has recently gone from a two step process of train and binarize to a single step simultaneous train/compress. Performance on a few small tasks show the benefit. It would be nice to see some results on substantial networks and tasks which really need compression on embedded systems (a point made in the introduction). Is it necessary to discuss exploding/vanishing gradients when the RNN experiments are carried out by an LSTM, and handled by the cell error carousel? We see the desire to tie into proposition 2, but not clear that the degradation we see in the binary connect is related. Adam is used in the LSTM optimization, was gradient clipping really needed, or is the degradation of binary connect simply related to capacity? For proposition 3.1, theorem 3.1 and proposition 3.2 put the pointers to proofs in appendix.
Output (English): en