Model distillation ?
#11
by
noecasas
- opened
Thanks for this awesome model. Do someone know of any github repo or Colab notebook to distill this large model into a smaller one?
Hey @noecasas ! As far as I'm aware, there isn't a ready implementation for distilling a sequence-to-sequence ASR model. It should be possible though through vanilla knowledge-distillation with a teacher-student configuration! I'm personally also interested in running this experiment!