Before revision:  In this work, we construct explicitly multi-timescale language models by manipulating the input and URLet gate biases in a long short-term memory (LSTM) network.
After revision:  In this work, we derived a theory for how the memory gating mechanism in long short-term memory (LSTM) network.