Why is there no intermediate checkpoint between 500B-1300B?
#11
by
siqi-zz
- opened
Why is there no intermediate checkpoint between 500B-1300B?
And missing checkpoints at the beginning
Can I please bump this question? Earlier checkpoints for 1B parameters and < 84B tokens would be much appreciated!
Sorry, these checkpoints are missing because of a screw-up. We lost a range of data. It's relatively easy to re-create those checkpoints, but it takes a little bit of compute.
@dirkgr thank you! Would it be possible to recreate them? Or tell me what's necessary to re-create them?