from other definitions of semi-supervised learning such as rotating an image and then predicting the rotation angle or something like word to vector wave defect where you mask out certain parts of the sequence and then train the model to predict the missing part of the sequence this idea of semi-supervised learning is to have a label data set such as the image net data set train a large capacity model like a res NEX 101 32 by 48 group convolutions res next architecture and then use this massive high capacity