TheBloke
/

medalpaca-13B-GPTQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

TheBloke commited on Apr 23, 2023

Commit

7fc0c22

•

1 Parent(s): c27f1be

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -30,7 +30,7 @@ Unless you are able to use the latest GPTQ-for-LLaMa code, please use `medalpaca
   * Works with text-generation-webui one-click-installers
   * Works on Windows
   * Parameters: Groupsize = 128g. No act-order.
-  * Command:
     ```
     CUDA_VISIBLE_DEVICES=0 python3 llama.py medalpaca-13b c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors medalpaca-13B-GPTQ-4bit-128g.no-act-order.safetensors
     ```
@@ -40,7 +40,7 @@ Unless you are able to use the latest GPTQ-for-LLaMa code, please use `medalpaca
   * **Does not** work on Windows
   * Parameters: Groupsize = 128g. act-order.
   * Offers highest quality quantisation, but requires recent GPTQ-for-LLaMa code
-  * Command:
     ```
     CUDA_VISIBLE_DEVICES=0 python3 llama.py medalpaca-13b c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors medalpaca-13B-GPTQ-4bit-128g.safetensors
     ```

   * Works with text-generation-webui one-click-installers
   * Works on Windows
   * Parameters: Groupsize = 128g. No act-order.
+  * Command used to create the GPTQ:
     ```
     CUDA_VISIBLE_DEVICES=0 python3 llama.py medalpaca-13b c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors medalpaca-13B-GPTQ-4bit-128g.no-act-order.safetensors
     ```
   * **Does not** work on Windows
   * Parameters: Groupsize = 128g. act-order.
   * Offers highest quality quantisation, but requires recent GPTQ-for-LLaMa code
+  * Command used to create the GPTQ:
     ```
     CUDA_VISIBLE_DEVICES=0 python3 llama.py medalpaca-13b c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors medalpaca-13B-GPTQ-4bit-128g.safetensors
     ```