Update README.md
Browse files
README.md
CHANGED
@@ -49,9 +49,9 @@ ASSISTANT:
|
|
49 |
|
50 |
## Quantization process:
|
51 |
|
52 |
-
| Original Model | β | Float16 Model | β | Safetensor Model | β | EXL2 Model |
|
53 |
| -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
|
54 |
-
| [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | β | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF) | β | Safetensor
|
55 |
|
56 |
Example to convert WizardLM-70B-V1.0-HF_float16_safetensored to EXL2 4.0 bpw with 6-bit head:
|
57 |
|
@@ -60,9 +60,13 @@ mkdir -p ~/EXL2/WizardLM-70B-V1.0-HF_4bit # Create the output directory
|
|
60 |
python convert.py -i ~/float16_safetensored/WizardLM-70B-V1.0-HF -o ~/EXL2/WizardLM-70B-V1.0-HF_4bit -c ~/EXL2/0000.parquet -b 4.0 -hb 6
|
61 |
```
|
62 |
|
63 |
-
\* Use
|
|
|
|
|
|
|
|
|
|
|
64 |
|
65 |
- https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)
|
66 |
- https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py (recommended if model already converted to float16)
|
67 |
-
- https://gist.github.com/epicfilemcnulty/1f55fd96b08f8d4d6693293e37b4c55e#file-2safetensors-py
|
68 |
-
- https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py (best for sharding and float16/FP16 or bfloat16/BF16 conversion)
|
|
|
49 |
|
50 |
## Quantization process:
|
51 |
|
52 |
+
| Original Model | β | Float16 Model* | β | Safetensor Model** | β | EXL2 Model |
|
53 |
| -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
|
54 |
+
| [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | β | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF)* | β | Safetensor** | β | EXL2 |
|
55 |
|
56 |
Example to convert WizardLM-70B-V1.0-HF_float16_safetensored to EXL2 4.0 bpw with 6-bit head:
|
57 |
|
|
|
60 |
python convert.py -i ~/float16_safetensored/WizardLM-70B-V1.0-HF -o ~/EXL2/WizardLM-70B-V1.0-HF_4bit -c ~/EXL2/0000.parquet -b 4.0 -hb 6
|
61 |
```
|
62 |
|
63 |
+
\* Use the following script to convert your local pytorch_model bin files to float16 (you can also choose bfloat16) + safetensors all in one go:
|
64 |
+
|
65 |
+
- https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py
|
66 |
+
(best for sharding and float16/FP16 or bfloat16/BF16 conversion)
|
67 |
+
|
68 |
+
\*\* Use any one of the following scripts to convert your local pytorch_model bin files to safetensors:
|
69 |
|
70 |
- https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)
|
71 |
- https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py (recommended if model already converted to float16)
|
72 |
+
- https://gist.github.com/epicfilemcnulty/1f55fd96b08f8d4d6693293e37b4c55e#file-2safetensors-py
|
|