Task: Summarization
Input (English): Long short-term memory networks (LSTMs) were introduced to combat vanishing gradients in simple recurrent neural networks (S-RNNs) by augmenting them with additive recurrent connections controlled by gates. We present an alternate view to explain the success of LSTMs: the gates themselves are powerful recurrent models that provide more representational power than previously appreciated. We do this by showing that the LSTM's gates can be decoupled from the embedded S-RNN, producing a restricted class of RNNs where the main recurrence computes an element-wise weighted sum of context-independent functions of the inputs. Experiments on a range of challenging NLP problems demonstrate that the simplified gate-based models work substantially better than S-RNNs, and often just as well as the original LSTMs, strongly suggesting that the gates are doing much more in practice than just alleviating vanishing gradients.
Output (English): This paper presents an analysis of LSTMS, showing that they have a form where the memory cell contents at each step is a weighted combination of the “content update” values computed at each time step and offers a simplification of LSTMs that compute the value by which the memory cell at each time step in terms of a deterministic function of the input rather than a function of the input and the current context.