TheBloke commited on
Commit
1952ac7
1 Parent(s): 58a9172

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -0
README.md CHANGED
@@ -67,6 +67,13 @@ And after it, add:
67
  [Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
68
 
69
  3. Then save and close the file, and launch text-generation-webui as described below
 
 
 
 
 
 
 
70
 
71
  ## How to download and use this model in text-generation-webui
72
 
 
67
  [Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
68
 
69
  3. Then save and close the file, and launch text-generation-webui as described below
70
+ 4. Install and compile AutoGPTQ from source - note that this requires compiling the CUDA kernel, which requires CUDA toolkit. This may be an issue for Windows users.
71
+
72
+ ```
73
+ git clone https://github.com/PanQiWei/AutoGPTQ
74
+ cd AutoGPTQ
75
+ pip install . # This step requires CUDA toolkit installed
76
+ ```
77
 
78
  ## How to download and use this model in text-generation-webui
79