common to use these two terms together semi-supervised learning and model distillation also interesting lis is we can see this kind of model compression that arises from this framework you can have a really high capacity teacher model like the res NEX 101 32 by 48 D and then you can have a lower capacity more manageable probably faster inference time lower storage costs like the res net 50 that you could deploy on mobile and IOT in these kinds of things this animation shows the extension from the semi-supervised training framework