Finetuning

#7
by Itamra - opened

I am trying to fine-tune the "OpenHathi-7B-Hi-v0.1-Base" model on a custom dataset. As the model is very large(6.2 Billion parameters), it can not be directly fine-tuned. Is there any documentation available for fine-tuning ?

Wondering the same thing. Any updates from the team?

Yes, we can train this OpenHathi model as LLAMA model using peft and lora.

Sign up or log in to comment