you interpret a particular token you can your context is only to the left of that token you don't know what's coming in it's like what do you when you read a sentence from left to right but then as humans unconsciously we probably go and at the end of the sentence kind of make sense of the thing as a whole but here the model is forced to make sense of what the thing only from whatever is to the left of it so that's a basic limitation of these left to right models then there is another approach which is called Elmo