Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
@@ -15,6 +15,12 @@ Credits to Ajay Jaiswal, Jinhao Duan, Zhenyu Zhang, Zhangheng Li, Lu Yin, Shiwei
|
|
15 |
|
16 |
License: [MIT License](https://opensource.org/license/mit/)
|
17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
Setup environment
|
19 |
```shell
|
20 |
pip install torch==2.0.0+cu117 torchvision==0.15.1+cu117 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu117
|
@@ -23,6 +29,8 @@ pip install accelerate
|
|
23 |
pip install auto-gptq # for gptq
|
24 |
```
|
25 |
|
|
|
|
|
26 |
How to use pruned models
|
27 |
```python
|
28 |
import torch
|
|
|
15 |
|
16 |
License: [MIT License](https://opensource.org/license/mit/)
|
17 |
|
18 |
+
Simplified lists:
|
19 |
+
* Models: Llama-2 13b, Llama-2 chat 13b, Vicuna 13b v1.3
|
20 |
+
* Compression methods:
|
21 |
+
- Pruning: Magnitude-based, Wanda, SparseGPT (2:4 semi-structured)
|
22 |
+
- Quantization: AWQ, GPTQ (3,4,8 bits)
|
23 |
+
|
24 |
Setup environment
|
25 |
```shell
|
26 |
pip install torch==2.0.0+cu117 torchvision==0.15.1+cu117 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu117
|
|
|
29 |
pip install auto-gptq # for gptq
|
30 |
```
|
31 |
|
32 |
+
## How to use models
|
33 |
+
|
34 |
How to use pruned models
|
35 |
```python
|
36 |
import torch
|