OPEA
/

Safetensors
qwen2
4-bit precision
intel/auto-round
cicdatopea commited on
Commit
aa98c01
·
verified ·
1 Parent(s): 38b23d7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -16,7 +16,7 @@ CPU/ CUDA requires auto-round version>0.3.1
16
  ```python
17
  from auto_round import AutoRoundConfig ##must import for auto-round format
18
  from transformers import AutoModelForCausalLM,AutoTokenizer
19
- quantized_model_dir = "Intel/Qwen2.5-32B-Instruct-int4-inc"
20
  tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir)
21
 
22
  model = AutoModelForCausalLM.from_pretrained(
@@ -127,7 +127,7 @@ prompt = "请简短介绍一下阿里巴巴公司"
127
  pip3 install lm-eval==0.4.5
128
 
129
  ```bash
130
- auto-round --model "Intel/Qwen2.5-32B-Instruct-int4-inc" --eval --eval_bs 16 --tasks leaderboard_ifeval,leaderboard_mmlu_pro,gsm8k,lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,cmmlu,ceval-valid
131
  ```
132
 
133
  | Metric | BF16 | INT4 |
 
16
  ```python
17
  from auto_round import AutoRoundConfig ##must import for auto-round format
18
  from transformers import AutoModelForCausalLM,AutoTokenizer
19
+ quantized_model_dir = "OPEA/Qwen2.5-32B-Instruct-int4-inc"
20
  tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir)
21
 
22
  model = AutoModelForCausalLM.from_pretrained(
 
127
  pip3 install lm-eval==0.4.5
128
 
129
  ```bash
130
+ auto-round --model "OPEA/Qwen2.5-32B-Instruct-int4-inc" --eval --eval_bs 16 --tasks leaderboard_ifeval,leaderboard_mmlu_pro,gsm8k,lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,cmmlu,ceval-valid
131
  ```
132
 
133
  | Metric | BF16 | INT4 |