Can't you make a finetune instead?
#1
by
TheYuriLover
- opened
All the Qloras in the airoboros llama1 were underwhelming compared than their finetuned counterpart.
Your llama1 airoboros 1.4 finetuned model is still my favorite model, please consider redoing it with llama2
It's expensive, and I pay out of pocket, so no, not at this time. The 2.0 dataset is nearly done, and for that I will do full fine tune again.
Oh ok cool, as long as you make a finetune at the end I'm fine with that. Good luck man o/
I guess that new dataset is gonna be more "lengthy" in the bot's output? Now we got 4k context we can make use of that.