bigscience

/

bloom-560m-intermediate

like 1

I tried to evaluate the intermediate checkpoint by perplexity on WMT21 zh-en translation test set. Specifically, I ran all checkpoints from 1000step to 600,000 step and the ppl on the English translation (only the English reference is fed into the model to calculate ppl) are 392, 71, 53, 51, 50, 50, 71, 320322 respectively. It seems that the 500,000step checkpoint is the same as the 10,000 step checkpoint and there are some errors with the 600,000 step checkpoint.