this pixel right here you only want information from whatever it was before that pixel otherwise the model could cheat right otherwise the model could simply learn to copy over the value but the attention pattern here is simply to show you that this is auto regressive and it's in one direction so you always want to predict the next pixel and then from all of this you want to predict the next pixel and from all of this you want to predict the next pixel this is in contrast to this objective here that comes from Bert and