Is this a float32 model?
#2
by
RonanMcGovern
- opened
The model is 27 GB so I assume it has been uploaded in float32, not bfloat16?
Yes. We use zero_to_fp32.py provided by DeepSpeed to convert from deepspeed weights to fp32 via
python3 zero_to_fp32.py . pytorch_model.bin
Thanks, yeah I noticed that other models you have are in bf16 so I was wondering about this one.
BTW, I'm working to make AWQs for these models, but hitting issues with AWQ right now, see github issue here
RonanMcGovern
changed discussion status to
closed