we've just do linear probing if you fine-tune these architectures then you reach like a 99% accuracy on C for ten which is on par with the best models that we have so G pipe is supervised pre trained on imagenet but also I guess uses a bunch of data element ation while these image GPT it uses minimal data augmentation I think they simply random crop a little bit and that's about it so they also experiment around with this Bert objective so until now this was all the this was all this autoregressive objective and I feel the open hi people