will there be a tinymistral?
#3
by
LaferriereJC
- opened
that would be nice.
I'm really looking for a 3b model, but I'll settle for 1.1b
You would need the Mistral dataset.
Otherwise someone can do a 'distillation'. Let Mistral blabber on, and train on that.
There might be better pruning methods though.