the probability distribution of sentences here X is a sentence right and this can be naturally factorized into a product over the words where the probability of each word is only dependent on the words before it so this is a this is an equality is not an approximation this is an equality the probability of a sequence can be decomposed into a product of probabilities like this exactly so this here is exactly what these auto regressive models implement each word is predicted from the words before it right there are other kinds of autoregressive