AQLM
Collection
AQLM quantized LLMs
•
20 items
•
Updated
•
46
Official AQLM quantization of meta-llama/Meta-Llama-3-70B-Instruct .
For this quantization, we used 1 codebook of 16 bits.
Results (measured with lm_eval==4.0):
| Model | Quantization | MMLU (5-shot) | ArcC | ArcE | Hellaswag | Winogrande | PiQA | Model size, Gb |
|---|---|---|---|---|---|---|---|---|
| meta-llama/Meta-Llama-3-70B | - | 0.7980 | 0.6160 | 0.8624 | 0.6367 | 0.8183 | 0.7632 | 141.2 |
| 1x16 | 0.7587 | 0.4863 | 0.7668 | 0.6159 | 0.7481 | 0.7537 | 21.9 |
- Downloads last month
- 349
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.