request for training notebook to finetune falcon 7b
hope you could share the .ipynb for fine-tuning the falcon-7b model.
it would be great if you could upload it as a colab notebook.
also have u explored QLora?
No I used axolotl.
https://github.com/OpenAccess-AI-Collective/axolotl
I don't need qLoRA for this small training job. When I train 65b then I'll use qLoRA.
Absolutely love this idea! Any chance you could release the training set, @ehartford ? :)
I'm working on it.
can you please describe how you managed to fine tune it ?
can you please describe how you managed to fine tune it ?
Do you mean his parameters to axolotl?
yes ! please.
I used this
Changed the file format to sharegpt:chat
https://github.com/OpenAccess-AI-Collective/axolotl/blob/main/examples/falcon/config-7b.yml
thank you !!
Did you use:
python scripts/finetune.py examples/falcon/config-7b.yml
or:
accelerate launch scripts/finetune.py examples/falcon/config-7b.yml