combines very well to transfer learning like Bert and scales for the model parameters and model sizes so they also experimented with imagenet where they take unlabeled imagenet of 1 million examples and 1 million unlabeled data points they use 10% label data which is 100,000 labels or 100 labels per image and pure supervised gets 55% whereas their model you DEA gets 68.9 close to 69 percent accuracy which is at least 30% better and this shows the benefits are using more unlabeled data points so another thing that they tried