Further train for another 1250K steps (since base) on CC-100, Twitter, Updated Wikipedia

Further train for another 800K steps on CC-100, Twitter, Updated Wikipedia

Saving weights and logs of step 10500

Saving weights and logs of step 9750

Saving weights and logs of step 9000

Saving weights and logs of step 8250

Saving weights and logs of step 7500

Saving weights and logs of step 6750

Saving weights and logs of step 6000

Saving weights and logs of step 5250

Saving weights and logs of step 4500

Saving weights and logs of step 3750

Saving weights and logs of step 3000

Saving weights and logs of step 2250

Saving weights and logs of step 1500

Saving weights and logs of step 750

Saving weights and logs of step 4500

Saving weights and logs of step 3750

Saving weights and logs of step 3000

Saving weights and logs of step 2250

Saving weights and logs of step 1500

Saving weights and logs of step 750

Saving weights and logs of step 3750

Saving weights and logs of step 2250

Saving weights and logs of step 750

Saving weights and logs of step 5250

Saving weights and logs of step 4500

Saving weights and logs of step 3750

Saving weights and logs of step 3000

Saving weights and logs of step 2250

Saving weights and logs of step 1500

Saving weights and logs of step 750

250 steps

After a few hours of training

Testing Jax model training. After a few iterations (still in the first epoch), testing model save

Testing JAX training with TPU VM, After 1 Epoch