Update README.md
Browse files
README.md
CHANGED
|
@@ -755,6 +755,28 @@ print(tokenizer.apply_chat_template(chat, tokenize=False))
|
|
| 755 |
```
|
| 756 |
</details>
|
| 757 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 758 |
## Support me
|
| 759 |
|
| 760 |
https://bmc.link/jondurbin
|
|
|
|
| 755 |
```
|
| 756 |
</details>
|
| 757 |
|
| 758 |
+
## Renting instances to run the model
|
| 759 |
+
|
| 760 |
+
### Latitude.sh
|
| 761 |
+
|
| 762 |
+
[Latitude](https://www.latitude.sh/r/4BBD657C) has h100 instances available (as of today, 2024-02-08) for $3/hr!
|
| 763 |
+
|
| 764 |
+
I've added a blueprint for running text-generation-webui within their container system:
|
| 765 |
+
https://www.latitude.sh/dashboard/create/containerWithBlueprint?id=7d1ab441-0bda-41b9-86f3-3bc1c5e08430
|
| 766 |
+
|
| 767 |
+
Be sure to set the following environment variables:
|
| 768 |
+
|
| 769 |
+
| key | value |
|
| 770 |
+
| --- | --- |
|
| 771 |
+
| PUBLIC_KEY | `{paste your ssh public key}` |
|
| 772 |
+
| UI_ARGS | `--trust-remote-code` |
|
| 773 |
+
|
| 774 |
+
Access the webui via `http://{container IP address}:7860`, navigate to model, download jondurbin/bagel-20b-v04, and ensure the following values are set:
|
| 775 |
+
|
| 776 |
+
- `use_flash_attention_2` should be checked
|
| 777 |
+
- set Model loader to Transformers
|
| 778 |
+
- `trust-remote-code` should be checked
|
| 779 |
+
|
| 780 |
## Support me
|
| 781 |
|
| 782 |
https://bmc.link/jondurbin
|