previous words predict the next word in the language model that's like what GPT one does and so what we see with that is you're not able to exploit context for the right you're not able to see you know you by masking the model you have to prevent it from being able to just look at the next word and say well I see my sequence that it's cat so I'll just learn to copy it over in predict cat so that has a major limitation and when we when we release GP t1 we actually like weren't able to do well on or you know we found that some of the