metadata

library_name: nanotron

⚙️ Nano-Mistral

Modeling code for Mistral to use with Nanotron

🚀 Quickstart

# Generate a config file
python config_tiny_mistral.py


# Run training
export CUDA_DEVICE_MAX_CONNECTIONS=1 # important for some distributed operations
torchrun --nproc_per_node=8 run_train.py --config-file config_tiny_mistral.yaml