wenhuach commited on
Commit
a71e3b7
·
verified ·
1 Parent(s): f01205c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -22
README.md CHANGED
@@ -15,28 +15,7 @@ This model is an int4 model with group_size 128 of [microsoft/phi-2](https://hug
15
 
16
  ## How To Use
17
 
18
- ### Reproduce the model
19
-
20
- Here is the sample command to reproduce the model
21
-
22
- ```bash
23
- git clone https://github.com/intel/auto-round
24
- cd auto-round/examples/language-modeling
25
- pip install -r requirements.txt
26
- python3 main.py \
27
- --model_name microsoft/phi-2 \
28
- --device 0 \
29
- --group_size 128 \
30
- --bits 4 \
31
- --iters 1000 \
32
- --enable_minmax_tuning \
33
- --deployment_device 'gpu' \
34
- --scale_dtype 'fp32' \
35
- --eval_bs 32 \
36
- --output_dir "./tmp_autoround" \
37
- --amp
38
 
39
- ```
40
 
41
 
42
 
@@ -84,6 +63,28 @@ Since we encountered an issue evaluating this model with lm-eval, we opted to ev
84
  | arc_challenge | 0.5290 | 0.5196 |
85
 
86
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
 
88
  ## Ethical Considerations and Limitations
89
 
@@ -102,4 +103,16 @@ Here are a couple of useful links to learn more about Intel's AI software:
102
 
103
  ## Disclaimer
104
 
105
- The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
  ## How To Use
17
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
 
 
19
 
20
 
21
 
 
63
  | arc_challenge | 0.5290 | 0.5196 |
64
 
65
 
66
+ ### Reproduce the model
67
+
68
+ Here is the sample command to reproduce the model
69
+
70
+ ```bash
71
+ git clone https://github.com/intel/auto-round
72
+ cd auto-round/examples/language-modeling
73
+ pip install -r requirements.txt
74
+ python3 main.py \
75
+ --model_name microsoft/phi-2 \
76
+ --device 0 \
77
+ --group_size 128 \
78
+ --bits 4 \
79
+ --iters 1000 \
80
+ --enable_minmax_tuning \
81
+ --deployment_device 'gpu' \
82
+ --scale_dtype 'fp32' \
83
+ --eval_bs 32 \
84
+ --output_dir "./tmp_autoround" \
85
+ --amp
86
+
87
+ ```
88
 
89
  ## Ethical Considerations and Limitations
90
 
 
103
 
104
  ## Disclaimer
105
 
106
+ The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.
107
+
108
+
109
+ ## Cite
110
+
111
+ @article{cheng2023optimize,
112
+ title={Optimize weight rounding via signed gradient descent for the quantization of llms},
113
+ author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao},
114
+ journal={arXiv preprint arXiv:2309.05516},
115
+ year={2023}
116
+ }
117
+
118
+ [arxiv](https://arxiv.org/abs/2309.05516) [github](https://github.com/intel/auto-round)