Conversion to pytorch
#1
by
alpindale
- opened
Thanks for the fix. Can you share the script you used for converting the f16 gguf to pytorch?
- build llama.cpp, run
./quantize --pure --allow-requantize ./miqu-1-70b.q5_K_M.gguf 1
- apply a.patch, run
python3 convert.py --dump ./miqu-1-70b.q5_K_M.gguf
- run https://gist.github.com/152334H/27d4181ce3641cec335131b971584ddd
152334H
changed discussion status to
closed
behaviour is hardcoded for llama2-70b shape so be weary of that much
also, the result of d = {v[1]:k for k,v in tm.get_tensor_name_map(tm.MODEL_ARCH.LLAMA, LAYERS).mapping.items()}
is an unstable hack that will change in future llama.cpp versions
Interesting. Now I'm praying that I can train/merge qlora and quant to exl2 without issues lol.