Checkpointing after getting some moe models in place.