Commit History

fix run.sh
6665f93

sakares commited on

use proper tokenizer.json and change to batch_size=256
de940b9

sakares commited on

increaaase bs=128 and change dtype to fp32
2071335

sakares commited on

remove old exp&model and lower steps to save
64934c7

sakares commited on

fix script
9c0b789

sakares commited on

make it resumable training
51e884f

sakares commited on

update script
8976790

sakares commited on

lower lr regarding explode gradient. also new seed number
b0729e9

sakares commited on

switch back to batch_size = 32
9307a78

sakares commited on

change saving and eval steps
e73a28c

sakares commited on

adjust batch_size and update run_mlm_flax script
6338ce3

sakares commited on

adjust running script
51a46c2

sakares commited on

reduce batch size
1176e2d

sakares commited on

change to small batch size
0f4f657

sakares commited on

change dtype to bfloat16
105f01c

sakares commited on

adjust script
c20670b

sakares commited on

update setting and batch size
6645c29

sakares commited on

change batch size and command
9cc0124

sakares commited on

add runing script
39aa441

sakares commited on