Hugging Face's logo

Thireus
/
WizardLM-70B-V1.0-HF-4.0bpw-h6-exl2

Text Generation
Transformers
llama
Model card Files Community
Thireus commited on
Commit
511017b
•
1 Parent(s): 64d594a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -45,17 +45,17 @@ ASSISTANT:
45
 
46
  | Original Model | → | Float16 Model | → | Safetensor Model | → | EXL2 Model |
47
  | -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
48
- | [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | → | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF) | → | Safetensor | → | EXL2 |
49
 
50
- Use any one of the following scripts to convert your float16 pytorch_model bin files to safetensors:
51
 
52
  - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py
53
  - https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py
54
  - https://gist.github.com/epicfilemcnulty/1f55fd96b08f8d4d6693293e37b4c55e
55
  - https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py
56
 
57
- Example to convert WizardLM-70B-V1.0-HF_float16_safetensored to EXL2 4.0 bpw with 6-bit head:
58
 
 
59
  ```
60
  mkdir -p ~/EXL2/WizardLM-70B-V1.0-HF_4bit # Create the output directory
61
  python convert.py -i ~/safetensor/WizardLM-70B-V1.0-HF_float16_safetensored -o ~/EXL2/WizardLM-70B-V1.0-HF_4bit -c ~/EXL2/0000.parquet -b 4.0 -hb 6
 
45
 
46
  | Original Model | → | Float16 Model | → | Safetensor Model | → | EXL2 Model |
47
  | -------------- | --- | ------------- | --- | ---------------- | --- | ---------- |
48
+ | [WizardLM 70B V1.0](https://huggingface.co/WizardLM/WizardLM-70B-V1.0) | → | [WizardLM 70B V1.0-HF](https://huggingface.co/simsim314/WizardLM-70B-V1.0-HF) | → | (*) Safetensor | → | EXL2 |
49
 
50
+ (*) Use any one of the following scripts to convert your float16 pytorch_model bin files to safetensors:
51
 
52
  - https://github.com/turboderp/exllamav2/blob/master/util/convert_safetensors.py
53
  - https://huggingface.co/Panchovix/airoboros-l2-70b-gpt4-1.4.1-safetensors/blob/main/bin2safetensors/convert.py
54
  - https://gist.github.com/epicfilemcnulty/1f55fd96b08f8d4d6693293e37b4c55e
55
  - https://github.com/oobabooga/text-generation-webui/blob/main/convert-to-safetensors.py
56
 
 
57
 
58
+ Example to convert WizardLM-70B-V1.0-HF_float16_safetensored to EXL2 4.0 bpw with 6-bit head:
59
  ```
60
  mkdir -p ~/EXL2/WizardLM-70B-V1.0-HF_4bit # Create the output directory
61
  python convert.py -i ~/safetensor/WizardLM-70B-V1.0-HF_float16_safetensored -o ~/EXL2/WizardLM-70B-V1.0-HF_4bit -c ~/EXL2/0000.parquet -b 4.0 -hb 6