Update README.md

README.md

@@ -42,6 +42,17 @@ This repo contains GPTQ model files for [Mistral AI's Mistral 7B Instruct v0.1](
 
 Multiple GPTQ parameter permutations are provided; see Provided Files below for details of the options provided, their parameters, and the software used to create them.
 
+### GPTQs will work in Transformers only - and requires Transformers from Github
+
+At the time of writing (September 28th), AutoGPTQ has not yet added support for the new Mistral models.
+
+These GPTQs were made directly from Transformers, and so can only be loaded via the Transformers interface.  They can't be loaded directly from AutoGPTQ.
+
+In addition, you will need to install Transformers from Github, with:
+```
+pip3 install git+https://github.com/huggingface/transformers.git@72958fcd3c98a7afdc61f953aa58c544ebda2f79
+```
+
 <!-- description end -->
 <!-- repositories-available start -->
 ## Repositories available
@@ -70,7 +81,7 @@ Multiple quantisation parameters are provided, to allow you to choose the best o
 
 Each separate quant is in a different branch.  See below for instructions on fetching from different branches.
 
-All recent GPTQ files are made with AutoGPTQ, and all files in non-main branches are made with AutoGPTQ. Files in the `main` branch which were uploaded before August 2023 were made with GPTQ-for-LLaMa.
+These files were made with Transformers 4.34.0.dev0, from commit 72958fcd3c98a7afdc61f953aa58c544ebda2f79.
 
 <details>
   <summary>Explanation of GPTQ parameters</summary>
@@ -164,6 +175,10 @@ Note that using Git with HF repos is strongly discouraged. It will be much slowe
 <!-- README_GPTQ.md-text-generation-webui start -->
 ## How to easily download and use this model in [text-generation-webui](https://github.com/oobabooga/text-generation-webui).
 
+NOTE: These models haven't been tested in text-generation-webui. But I hope they will work.
+
+You will need to use **Loader: Transformers**.  AutoGPTQ will not work.  I don't know about ExLlama - it might work as this model is so similar to Llama; let me know if it does!
+
 Please make sure you're using the latest version of [text-generation-webui](https://github.com/oobabooga/text-generation-webui).
 
 It is strongly recommended to use the text-generation-webui one-click-installers unless you're sure you know how to make a manual install.
@@ -187,10 +202,11 @@ It is strongly recommended to use the text-generation-webui one-click-installers
 
 ### Install the necessary packages
 
-Requires: Transformers 4.33.0 or later, Optimum 1.12.0 or later, and AutoGPTQ 0.4.2 or later.
+Requires: Transformers 4.34.0.dev0 from Github source, Optimum 1.12.0 or later, and AutoGPTQ 0.4.2 or later.
 
 ```shell
-pip3 install transformers optimum
+pip3 install optimum
+pip3 install git+https://github.com/huggingface/transformers.git@72958fcd3c98a7afdc61f953aa58c544ebda2f79
 pip3 install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/  # Use cu117 if on CUDA 11.7
 ```
 
@@ -251,11 +267,8 @@ print(pipe(prompt_template)[0]['generated_text'])
 <!-- README_GPTQ.md-compatibility start -->
 ## Compatibility
 
-The files provided are tested to work with AutoGPTQ, both via Transformers and using AutoGPTQ directly. They should also work with [Occ4m's GPTQ-for-LLaMa fork](https://github.com/0cc4m/KoboldAI).
-
-[ExLlama](https://github.com/turboderp/exllama) is compatible with Llama models in 4-bit. Please see the Provided Files table above for per-file compatibility.
+The files provided are only tested to work with Transformers 4.34.0.dev0 as of commit 72958fcd3c98a7afdc61f953aa58c544ebda2f79.
 
-[Huggingface Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference) is compatible with all GPTQ models.
 <!-- README_GPTQ.md-compatibility end -->
 
 <!-- footer start -->