Thireus commited on
Commit
71ec21e
β€’
1 Parent(s): 5cfe9bb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -5
README.md CHANGED
@@ -49,9 +49,9 @@ ASSISTANT:
49
 
50
  ## Quantization process:
51
 
52
- | Original Model | β†’ | Float16 Model | β†’ | Safetensor Model | β†’ | EXL2 Model |
53
  | -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
54
- | [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | β†’ | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF) | β†’ | Safetensor* | β†’ | EXL2 |
55
 
56
  Example to convert WizardLM-70B-V1.0-HF_float16_safetensored to EXL2 4.0 bpw with 6-bit head:
57
 
@@ -60,9 +60,13 @@ mkdir -p ~/EXL2/WizardLM-70B-V1.0-HF_4bit # Create the output directory
60
  python convert.py -i ~/float16_safetensored/WizardLM-70B-V1.0-HF -o ~/EXL2/WizardLM-70B-V1.0-HF_4bit -c ~/EXL2/0000.parquet -b 4.0 -hb 6
61
  ```
62
 
63
- \* Use any one of the following scripts to convert your local pytorch_model bin files to safetensors:
 
 
 
 
 
64
 
65
  - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)
66
  - https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py (recommended if model already converted to float16)
67
- - https://gist.github.com/epicfilemcnulty/1f55fd96b08f8d4d6693293e37b4c55e#file-2safetensors-py
68
- - https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py (best for sharding and float16/FP16 or bfloat16/BF16 conversion)
 
49
 
50
  ## Quantization process:
51
 
52
+ | Original Model | β†’ | Float16 Model* | β†’ | Safetensor Model** | β†’ | EXL2 Model |
53
  | -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
54
+ | [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | β†’ | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF)* | β†’ | Safetensor** | β†’ | EXL2 |
55
 
56
  Example to convert WizardLM-70B-V1.0-HF_float16_safetensored to EXL2 4.0 bpw with 6-bit head:
57
 
 
60
  python convert.py -i ~/float16_safetensored/WizardLM-70B-V1.0-HF -o ~/EXL2/WizardLM-70B-V1.0-HF_4bit -c ~/EXL2/0000.parquet -b 4.0 -hb 6
61
  ```
62
 
63
+ \* Use the following script to convert your local pytorch_model bin files to float16 (you can also choose bfloat16) + safetensors all in one go:
64
+
65
+ - https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py
66
+ (best for sharding and float16/FP16 or bfloat16/BF16 conversion)
67
+
68
+ \*\* Use any one of the following scripts to convert your local pytorch_model bin files to safetensors:
69
 
70
  - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py (official ExLlamaV2)
71
  - https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py (recommended if model already converted to float16)
72
+ - https://gist.github.com/epicfilemcnulty/1f55fd96b08f8d4d6693293e37b4c55e#file-2safetensors-py