Before revision:  Further, information about high-,mid-, and low-frequency words is routed preferentially through units with the appropriate timescales. Thus we show how to construct language modelswith interpretable representations of different information timescales .
After revision:  Further, information about high-,mid-, and low-frequency words is routed preferentially through units with different timescales, potentially improving model interpretability. These results demonstrate the importance of careful, theoretically-motivated analysis of memory and timescale in language models .