Are your merges all done in fp16?

#2
by ddh0 - opened

Hello! I appreciate the model. Was it merged from the full fp16 weights of each model, or quantized versions? Thanks.

Hello!
Merge can only be done with the full fp16.
You can't merge quantized model!
If you merge, make Llama2 model float16 and mistral model bfloat16

Thanks for the reply! That's what I thought, but I wanted to make sure. By the way, I merged dolphin-2.1-mistral with Toppy-M for good results. I have the original files uploaded and I'm uploading the GGUFs right now.

Sign up or log in to comment