Update README.md

README.md

@@ -20,29 +20,32 @@ I have the following Koala model repositories available:
 * [GPTQ quantized 4bit 7B model in `pt` and `safetensors` formats](https://huggingface.co/TheBloke/koala-7B-GPTQ-4bit-128g)
 * [GPTQ quantized 4bit 7B model in GGML format for `llama.cpp`](https://huggingface.co/TheBloke/koala-7B-GPTQ-4bit-128g-GGML)
 
-## Quantization method
-
-This GPTQ model was quantized using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) with the following commands:
-```
-python3 llama.py /content/koala-7B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save /content/koala-7B-4bit-128g.pt
-python3 llama.py /content/koala-7B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors /content/koala-7B-4bit-128g.safetensors
-```
-
-I used the latest Triton branch of `GPTQ-for-LLaMa` but they can also be loaded with the CUDA branch.
-
 ## Provided files
 
-I have provided both a `pt` and `safetensors` file. Either should work.
-
-If both are present in the model directory for text-generation-webui I am not sure which it chooses, so you may want to place only one in the models folder.
-
-The `olderFormat` file was created with the aim of then converting it to GGML for use with [llama.cpp](https://github.com/ggerganov/llama.cpp).  At present this file does not work.
-
-## How to run with `text-generation-webui`
-
-GPTQ model files provided will not load as-is with [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
-
-These model files require the latest version of the GPTQ code.
+Three model files are provided. You don't need all three - choose the one that suits your needs best!
+
+Details of the files provided:
+* `koala-7B-4bit-128g.pt`
+  * pt format file, created with the latest [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) code.
+  * Command to create:
+    * `python3 llama.py koala-7B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save koala-7B-4bit-128g.pt`
+* `koala-7B-4bit-128g.safetensors`
+  * newer `safetensors` format, with improved file security, created with the latest [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) code.
+  * Command to create:
+    * `python3 llama.py koala-7B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors koala-7B-4bit-128g.safetensors`
+* `koala-7B-4bit-128g.no-act-order.ooba.pt`
+  * `pt` format file, created with [oobabooga's older CUDA fork of GPTQ-for-LLaMa](https://github.com/oobabooga/GPTQ-for-LLaMa).
+  * This file is included primarily for Windows users, as it can be used without needing to compile the latest GPTQ-for-LLaMa code.
+  * It should hopefully therefore work with one-click-installers on Windows, which include the older GPTQ-for-LLaMa code.
+  * The older GPTQ code does not support all the latest features, so the quality may be fractionally lower.
+  * Command to create:
+    * `python3 llama.py koala-7B-HF c4 --wbits 4 --true-sequential --groupsize 128 --save koala-7B-4bit-128g.no-act-order.ooba.pt`
+      
+## How to run in `text-generation-webui`
+
+File `koala-7B-4bit-128g.no-act-order.ooba.pt` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
+
+The other two model files were created with the latest GPTQ code, and require that the latest GPTQ-for-LLaMa is used inside the UI.
 
 Here are the commands I used to clone the Triton branch of GPTQ-for-LLaMa, clone text-generation-webui, and install GPTQ into the UI:
 ```
@@ -55,12 +58,12 @@ ln -s GPTQ-for-LLaMa text-generation-webui/repositories/GPTQ-for-LLaMa
 Then install this model into `text-generation-webui/models` and launch the UI as follows:
 ```
 cd text-generation-webui
-python server.py --model koala-7B-4bit-128g --wbits 4 --groupsize 128 --model_type Llama # add any other command line args you want
+python server.py --model koala-13B-GPTQ-4bit-128g --wbits 4 --groupsize 128 --model_type Llama # add any other command line args you want
 ```
 
 The above commands assume you have installed all dependencies for GPTQ-for-LLaMa and text-generation-webui. Please see their respective repositories for further information.
 
-If you cannot use the Triton branch of GPTQ for any reason, you can alternatively use the CUDA branch instead:
+If you are on Windows, or cannot use the Triton branch of GPTQ for any other reason, you can instead use the CUDA branch:
 ```
 git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa -b cuda
 cd GPTQ-for-LLaMa
@@ -68,6 +71,8 @@ python setup_cuda.py install
 ```
 Then link that into `text-generation-webui/repositories` as described above.
 
+Or just use `koala-7B-4bit-128g.no-act-order.ooba.pt` as mentioned above.
+
 ## How the Koala delta weights were merged
 
 The Koala delta weights were originally merged using the following commands, producing [koala-7B-HF](https://huggingface.co/TheBloke/koala-7B-HF):