Training script for cosmo-1b?
#6
by
unnir
- opened
Is there a training code for cosmo-1d?
We used an internal wrapper around nanotron library https://github.com/huggingface/nanotron/ you can adapt this script https://github.com/loubnabnl/nanotron-smol-cluster/blob/main/brrr/cosmopedia/cosmo_1b.yaml