ISTA-DASLab
/

Mistral-7B-Instruct-v0.2-AQLM-2Bit-2x8

Text Generation

text-generation-inference

Inference Endpoints

8-bit precision

Model card Files Files and versions Community

SpiridonSunRotator commited on Apr 13

Commit

7f3acf3

•

1 Parent(s): a433cb8

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -14,5 +14,5 @@ For this quantization, we used 2 codebooks of 8 bits.
 Results:
 | Model      | Quantization | MMLU (5-shot) | Model size, Gb |
 |------|------|------|------|
-|CohereForAI/c4ai-command-r-v01| None | 0.5912 | 14.5 |
 |  | 2x8 | 0.4384 |  2.3 |

 Results:
 | Model      | Quantization | MMLU (5-shot) | Model size, Gb |
 |------|------|------|------|
+|mistralai/Mistral-7B-Instruct-v0.2 | None | 0.5912 | 14.5 |
 |  | 2x8 | 0.4384 |  2.3 |