are a bit more of a fan of the autoregressive objective just given what they've done so far in their papers and you can see here comparison of the two objectives on C for 10 and on image net again C for 10 is pre trained with image net and image net itself is pre trained with like a larger collection of images from the web all the pre training is done without labels now the blue is what you can reach with a linear probe and the orange is then on top of that what you can reach by fine-tuning okay so no linear profile tuning oh I have to say