context everything that's not masked in the context and this is only an approximate factorization because you're basically dropping away all these mask tokens and um what they do now is they do the same as the AR as their auto regressive models here they decompose to log probability into a sum of log from abilities over each of the words given all the words before it but now not before it in the sequence but before it in and chosen permutations Z and Z is sampled uniformly from the set of all possible permutations so in expectation