Initial AutoGPTQ model commit.
Browse files
README.md
CHANGED
|
@@ -34,15 +34,19 @@ I have [opened a PR here](https://github.com/oobabooga/text-generation-webui/pul
|
|
| 34 |
To get it working before the PR is merged, you will need to:
|
| 35 |
1. Edit `text-generation-webui/modules/AutoGPTQ_loader.py`
|
| 36 |
2. Make the following change:
|
|
|
|
| 37 |
Find the line that says:
|
| 38 |
```
|
| 39 |
'use_safetensors': use_safetensors,
|
| 40 |
```
|
|
|
|
| 41 |
And after it, add:
|
| 42 |
```
|
| 43 |
'trust_remote_code': shared.args.trust_remote_code,
|
| 44 |
```
|
|
|
|
| 45 |
[Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
|
|
|
|
| 46 |
3. Then save and close the file, and launch text-generation-webui as described below
|
| 47 |
|
| 48 |
## How to download and use this model in text-generation-webui
|
|
|
|
| 34 |
To get it working before the PR is merged, you will need to:
|
| 35 |
1. Edit `text-generation-webui/modules/AutoGPTQ_loader.py`
|
| 36 |
2. Make the following change:
|
| 37 |
+
|
| 38 |
Find the line that says:
|
| 39 |
```
|
| 40 |
'use_safetensors': use_safetensors,
|
| 41 |
```
|
| 42 |
+
|
| 43 |
And after it, add:
|
| 44 |
```
|
| 45 |
'trust_remote_code': shared.args.trust_remote_code,
|
| 46 |
```
|
| 47 |
+
|
| 48 |
[Once you are done the file should look like this](https://github.com/oobabooga/text-generation-webui/blob/473a57e35219c063d2fc230cfc7b5a118b448b38/modules/AutoGPTQ_loader.py#L33-L39)
|
| 49 |
+
|
| 50 |
3. Then save and close the file, and launch text-generation-webui as described below
|
| 51 |
|
| 52 |
## How to download and use this model in text-generation-webui
|