What are 0..7.bin?

by lozhnikov - opened

Hi guys! Thanks for this awesome work!
Could you please help me to understand what are those huge files taking up approximately 70 GB given that the model is 1B params of 16-bit precision? If my calculations are correct the plain weight file should be 2B bytes (β‰ˆ2 GB).

BigCode org

StarCoder is a 15B parameter model. You're probably thinking of SantaCoder, which is 1B. :)

@lozhnikov The model is 15.5 B params.
The 1B model is santacoder. You are looking at starcoder instead.
Also, weights are stored in 32-bit.

lozhnikov changed discussion status to closed

Sign up or log in to comment