smol llama
Collection
🚧"raw" pretrained smol_llama checkpoints - WIP 🚧
•
4 items
•
Updated
•
5
model card WIP, more details to come
A small 220M param (total) decoder model. This is the first version of the model.
Here are some fine-tunes we did, but there are many more possibilities out there!
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 29.44 |
AI2 Reasoning Challenge (25-Shot) | 24.83 |
HellaSwag (10-Shot) | 29.76 |
MMLU (5-Shot) | 25.85 |
TruthfulQA (0-shot) | 44.55 |
Winogrande (5-shot) | 50.99 |
GSM8k (5-shot) | 0.68 |