how you did conversion of omni models to 4/8bits?
#2
by
Jilt
- opened
I would like to convert some omni models to 4 bits using bnb. I am getting deepcopy error when trying with naive approach. Can you share with me the script that used for conversion of fp16 to 4 bits?
I use a very naive method too. The script can be found here: https://github.com/2dameneko/ide-cap-chan/blob/main/convert_to_bnb_nf4.py
You can customize it to your needs.