they learn weighted averages of the word vector so maybe for some low-level tasks you only want those input representations but maybe for some tasks you really want that kind of long-range context and so you might want to use the higher-level layers and so then they rear nth instead of feeding in those kind of like one-to-one look up in a table what the word vector is they have this RNN language model that processes the sentence or a piece of text in both directions and it learns to you know reuse its hidden state representation as