Heads up: this code does not actually perform distillation.
#2
by ryanontheinside - opened
It never runs the MiniMax DiT or a teacher model, uses no training data, and learns against a hand-written function of random noise. The uploaded LoRA confirms the problem: all 144 lora_up tensors are zero, making the adapter completely inert.
Hi @ryanontheinside ,
First off, I want to offer my sincere apologies for the initial prototype script and the inert adapter, and give you a HUGE thank you for pointing this out with such technical precision! 🙏
Your feedback was 100% spot-on. I immediately took everything down to rebuild the distillation engine completely from scratch. It is now 100% functional and tested.