---
license: apache-2.0
datasets:
- togethercomputer/RedPajama-Data-1T-Sample
language:
- en
---

This is another training run of [SmolLlamix-8x101M](https://huggingface.co/chargoddard/SmolLlamix-8x101M) with slightly different hyperparameters. Just testing to see how it holds up against the first run.