Is it fine to do granite-20b model's inference with bfloat16 dtype?
#1
by
lkm1
- opened
I have tried this model's inference with bfloat16 type and have not faced any problems. Is it fine to use bfloat16 dtype for this model's inference, without much effect on score?
we have run scores in bf16
lkm1
changed discussion status to
closed