Hugging Face's logo

Intel
/
Mistral-7B-v0.1-int4-inc

Text Generation
Transformers
Safetensors
mistral
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
wenhuach commited on
Commit
76e52ca
•
1 Parent(s): 2b928ed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -64,7 +64,7 @@ Install [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness.gi
64
  lm_eval --model hf --model_args pretrained="Intel/Mistral-7B-v0.1-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 128
65
  ```
66
 
67
- | Metric | FP16 | INT4 |
68
  | -------------- | ------ | ------ |
69
  | Avg. | 0.6306 | 0.6308 |
70
  | mmlu | 0.5961 | 0.5880 |
 
64
  lm_eval --model hf --model_args pretrained="Intel/Mistral-7B-v0.1-int4-inc",autogptq=True,gptq_use_triton=True --device cuda:0 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,rte,arc_easy,arc_challenge,mmlu --batch_size 128
65
  ```
66
 
67
+ | Metric | BF16 | INT4 |
68
  | -------------- | ------ | ------ |
69
  | Avg. | 0.6306 | 0.6308 |
70
  | mmlu | 0.5961 | 0.5880 |