pre training on unlabeled data like this whole data set without the labels can be a very good pre conditioner for fine-tuning later and this paper says the same thing so basically in in the good old days what you would do is you would devise a method that somehow takes you know takes in a devise a method that takes in a mini batch and in the mini batch you have your data samples and then some of them would be labeled right here you'd have Y and here it'd have a Y but most of them would be not labeled and you'd