they have a billion images from Instagram that they're using to train this model the fourth idea is really interesting that they use a large number of training iterations for their pre training with the weekly supervised learning compared to you know more pre-training iterations compared to normal supervised learning the fifth idea is a novel contribution to this paper the idea of having a balanced distribution for inferred labels so when you're doing the model distillation you want you don't want to have class and