| library_name: nanotron | |
| # ⚙️ Nano-Mistral | |
| Modeling code for Mistral to use with [Nanotron](https://github.com/huggingface/nanotron/) | |
| ## 🚀 Quickstart | |
| ```python | |
| # Generate a config file | |
| python config_tiny_mistral.py | |
| # Run training | |
| export CUDA_DEVICE_MAX_CONNECTIONS=1 # important for some distributed operations | |
| torchrun --nproc_per_node=8 run_train.py --config-file config_tiny_mistral.yaml | |
| ``` | |