Neural network based methods have obtained great progress on a variety of natural language processing tasks. However, it is still a challenge task to model long texts, such as sentences and documents. In this paper, we propose a multi-timescale long short-term memory (MT-LSTM) neural network to model long texts. MT-LSTM partitions the hidden states of the standard LSTM into several groups. Each group is activated at different time periods. Thus, MT-LSTM can model very long documents as well as short sentences. Experiments on four benchmark datasets show that our model outperforms the other neural models in text classification task.
Liu et al. REF introduced a Multi-Timescale Long Short-Term Memory (MT-LSTM) neural network to model short or long texts.
Multi-Timescale Long Short-Term Memory Neural Network for Modelling Sentences and Documents
EMNLP
Computer Science