metadata
license: gemma
| Wiki | C4 | PIQA | ARC-E | ARC-C | HellaSwag | Wino | Avg. | |
|---|---|---|---|---|---|---|---|---|
| Unquantized | 6.88 | 10.12 | 81.39 | 87.37 | 61.35 | 61.23 | 74.27 | 73.12 |
| W4G64 | 6.49 | 10.35 | 81.28 | 86.24 | 59.30 | 60.40 | 75.30 | 72.50 |
| W3G64 | 7.06 | 11.14 | 80.52 | 83.16 | 55.46 | 58.28 | 72.69 | 70.02 |
Revisions available in this repository:
main(W4G64, scales learned);nfl_w3g64(W3G64, scales learned);
Evaluations are provided for models with learned scales.
Benchmark scores (zero-shot) are computed with lm-evaluation-harness.