kuleshov

/

llama-7b-4bit

like 13

Hello,
Thanks for the model.

Do you think it is worth to finetune this version (just 4bit), and is there any example in notebook how to do so, even iI think it is pretty similar to GPT2 code for fine tuning.

I guess it is much better to finetune model in fp16 and then quantize it.

Thanks