n1ck-guo commited on
Commit
579c80b
·
verified ·
1 Parent(s): 1919913

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -6,7 +6,7 @@ datasets:
6
 
7
  ## Model Details
8
 
9
- This model is an int4 model with group_size 128 with quantized lm-head of [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model
10
 
11
  ## How To Use
12
 
@@ -78,7 +78,7 @@ print(response)
78
 
79
  ### Evaluate the model
80
 
81
- pip3 install lm-eval==0.4.2
82
 
83
  ```bash
84
  git clone https://github.com/intel/auto-round
@@ -87,7 +87,7 @@ python -m auto_round --model "Intel/Qwen2.5-3B-Instruct-int4-inc" --eval --eval_
87
  ```
88
 
89
  | Metric | BF16 | INT4(group_size 128) | INT4(group_size 32) |
90
- |:--------------:| :----: | :----: |:------:|
91
  | Avg | 0.5902 | 0.5879 | 0.5837 |
92
  | mmlu | 0.6546 | 0.6425 | 0.6454 |
93
  | cmmlu | 0.7223 | 0.6998 | 0.7069 |
 
6
 
7
  ## Model Details
8
 
9
+ This model is an int4 model with group_size 128 of [Qwen/Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model
10
 
11
  ## How To Use
12
 
 
78
 
79
  ### Evaluate the model
80
 
81
+ pip3 install lm-eval==0.4.4
82
 
83
  ```bash
84
  git clone https://github.com/intel/auto-round
 
87
  ```
88
 
89
  | Metric | BF16 | INT4(group_size 128) | INT4(group_size 32) |
90
+ |:-------------- | :----: | :----: |:------:|
91
  | Avg | 0.5902 | 0.5879 | 0.5837 |
92
  | mmlu | 0.6546 | 0.6425 | 0.6454 |
93
  | cmmlu | 0.7223 | 0.6998 | 0.7069 |