Before revision:  First, we show that initially training the model on short subsequences , before moving on to longer ones , both reduces overall training time and, surprisingly, gives a large improvement in perplexity.
After revision:  First, we show that initially training the model on short subsequences before moving on to longer ones , both reduces overall training time and, surprisingly, gives a large improvement in perplexity.