tokens: Looking at the state - of - the - art ( SOTA ) results , there is a pattern : the best performing models on small datasets like PTB and WikiText-2 are LSTM - based while the best performing models on larger datasets like WikiText-103 and 1Billion are dominated by Transformer models ( See Figures and ) .