Simingh commited on
Commit
9769312
1 Parent(s): 34f0aa8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -37,5 +37,6 @@ This repository contains all the intermediate checkpoints of OpenCoder-1.5B-Base
37
  - `anneal_iter_0001000` - `anneal_iter_0011920`: Intermediate checkpoints during the annealing stage.
38
 
39
 
40
- The number in each branch name indicates the corresponding current training step, where each step consumes 419,4304 training tokens (2,048 batch size * 4,096 sequence length from `pretrain_iter_0001000` - `pretrain_iter_0130000`. 1,024 batch size * 8,192 sequence length from `pretrain_iter_0001000` - `pretrain_iter_0130000`).\
 
41
  We use `pretrain_iter_0300000` as the starting point for the annealing stage, and use `anneal_iter_0010000` as the final base model.
 
37
  - `anneal_iter_0001000` - `anneal_iter_0011920`: Intermediate checkpoints during the annealing stage.
38
 
39
 
40
+ The number in each branch name indicates the corresponding current training step, where each step consumes 419,4304 training tokens (2,048 batch size * 4,096 sequence length from `pretrain_iter_0001000` - `pretrain_iter_0130000`. 1,024 batch size * 8,192 sequence length from `pretrain_iter_0001000` - `pretrain_iter_0130000`).
41
+
42
  We use `pretrain_iter_0300000` as the starting point for the annealing stage, and use `anneal_iter_0010000` as the final base model.