---
license: gemma
---

|             | Wiki | C4    | PIQA  | ARC-E | ARC-C | HellaSwag | Wino  | Avg.  |
| ----------- | ---- | ----- | ----- | ----- | ----- | --------- | ----- | ----- |
| Unquantized | 6.88 | 10.12 | 81.39 | 87.37 | 61.35 | 61.23     | 74.27 | 73.12 |
| W4G64       | 6.49 | 10.35 | 81.28 | 86.24 | 59.30 | 60.40     | 75.30 | 72.50 |
| W3G64       | 7.06 | 11.14 | 80.52 | 83.16 | 55.46 | 58.28     | 72.69 | 70.02 |

Revisions available in this repository:
- `main` (W4G64, scales learned);
- `nfl_w3g64` (W3G64, scales learned);

Evaluations are provided for models with learned scales.<br>Benchmark scores (zero-shot) are computed with [`lm-evaluation-harness`](https://github.com/EleutherAI/lm-evaluation-harness).