Are the weights in float32 of float16 ?
#18
by
benjamin-breton-loreal
- opened
Are the weights in float32 of float16 ? Saving them in float16 could reduce load time.
benjamin-breton-loreal
changed discussion title from
Are the weights in float32 of float16 ? Saving them in float16 could reduce load time.
to Are the weights in float32 of float16 ?
They're saved in 16-bit (12B params, 24GB on disk, you can see). You can load as fp16 or bf16.
srowen
changed discussion status to
closed