Anthonyg5005
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -48,6 +48,52 @@ There are a couple places you can use an exl2 model, here are a few:
|
|
48 |
- When using the downloader, make sure to format like this: Anthonyg5005/rishiraj-meow-10.7B-exl2**\:QuantBranch**
|
49 |
- [KoboldAI](https://github.com/henk717/KoboldAI) (Clone repo, don't use snapshot)
|
50 |
|
51 |
-
|
52 |
|
53 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
48 |
- When using the downloader, make sure to format like this: Anthonyg5005/rishiraj-meow-10.7B-exl2**\:QuantBranch**
|
49 |
- [KoboldAI](https://github.com/henk717/KoboldAI) (Clone repo, don't use snapshot)
|
50 |
|
51 |
+
# How to download:
|
52 |
|
53 |
+
### oobabooga's downloader
|
54 |
+
|
55 |
+
use something like [download-model.py](https://github.com/oobabooga/text-generation-webui/blob/main/download-model.py) to download with python requests.\
|
56 |
+
Install requirements:
|
57 |
+
|
58 |
+
```shell
|
59 |
+
pip install requests tqdm
|
60 |
+
```
|
61 |
+
|
62 |
+
Example for downloading 5bpw:
|
63 |
+
|
64 |
+
```shell
|
65 |
+
python download-model.py Anthonyg5005/rishiraj-meow-10.7B-exl2:5bit
|
66 |
+
```
|
67 |
+
|
68 |
+
### huggingface-cli
|
69 |
+
|
70 |
+
You may also use huggingface-cli\
|
71 |
+
To install it, install python hf-hub
|
72 |
+
|
73 |
+
```shell
|
74 |
+
pip install huggingface-hub
|
75 |
+
```
|
76 |
+
|
77 |
+
Example for 5bpw:
|
78 |
+
|
79 |
+
```shell
|
80 |
+
huggingface-cli download Anthonyg5005/rishiraj-meow-10.7B-exl2 --local-dir rishiraj-meow-10.7B-exl2-5bpw --revision 5bit
|
81 |
+
```
|
82 |
+
### Git LFS (not recommended)
|
83 |
+
|
84 |
+
I would recommend the http downloaders over using git, they can resume downloads if failed and are much easier to work with.\
|
85 |
+
Make sure to have git and git LFS installed.\
|
86 |
+
Example for 5bpw download with git:
|
87 |
+
|
88 |
+
Have LFS file skip disabled
|
89 |
+
```shell
|
90 |
+
# windows
|
91 |
+
set GIT_LFS_SKIP_SMUDGE=0
|
92 |
+
# linux
|
93 |
+
GIT_LFS_SKIP_SMUDGE=0
|
94 |
+
```
|
95 |
+
|
96 |
+
Clone repo branch
|
97 |
+
```shell
|
98 |
+
git clone https://huggingface.co/Anthonyg5005/rishiraj-meow-10.7B-exl2 -b 5bit
|
99 |
+
```
|