ISTA-DASLab
/

Meta-Llama-3-8B-AQLM-2Bit-1x16

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

SpiridonSunRotator commited on Jul 2

Commit

812d023

•

1 Parent(s): 2d9717e

Swapped PiQA and Winogrande results

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -14,7 +14,7 @@ Official [AQLM](https://arxiv.org/abs/2401.06118) quantization of [meta-llama/Me
 For this quantization, we used 1 codebook of 16 bits.
 Results:
-| Model      | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | Winogrande | PiQA | Model size, Gb |
 |------|------|-------|------|------|------|------|------|------|
 |meta-llama/Meta-Llama-3-8B| None |0.6525	| 0.5043 | 0.8009 | 0.6019 | 0.7971	| 0.7261 | 16.1 |
 |  | 1x16 | 0.5647 | 0.4130	| 0.7428 | 0.5545 | 0.7780 | 0.7198 | 4.1 |

 For this quantization, we used 1 codebook of 16 bits.
 Results:
+| Model      | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | PiQA | Winogrande | Model size, Gb |
 |------|------|-------|------|------|------|------|------|------|
 |meta-llama/Meta-Llama-3-8B| None |0.6525	| 0.5043 | 0.8009 | 0.6019 | 0.7971	| 0.7261 | 16.1 |
 |  | 1x16 | 0.5647 | 0.4130	| 0.7428 | 0.5545 | 0.7780 | 0.7198 | 4.1 |