their guideline that having massive unlabeled data sets is key to this framework this plot shows how the accuracy improves as a function of the number of pre training iterations so as stated in their recommendations they use a much larger amount of pre training epochs than supervised learning epochs really interesting Lee is there showing four billion training iterations it achieves the highest accuracy in this plot then they show the results of increasing the K parameter so the K parameter shown here is this idea of