model size
#3
by
HeHeYeast
- opened
Why is the model this big, as it has a size of 27G, which is often appear in 13B models.
@HeHeYeast it is stored in fp32 precision instead of fp16 precision which models are usually stored in.
fp32 precision is 2x bigger and 2x slower but you can easily convert it to fp16 precision by
model.to(dtype=torch.float16)