Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,39 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
license: mit
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
tags:
|
| 5 |
+
- ggml
|
| 6 |
+
- causal-lm
|
| 7 |
+
- gpt2
|
| 8 |
license: mit
|
| 9 |
---
|
| 10 |
+
```
|
| 11 |
+
▄▄▄ ██▓ ▓█████▄ █ ██ ███▄ █ ▄████ ▓█████ ▒█████ ███▄ █
|
| 12 |
+
▒████▄ ▓██▒ ▒██▀ ██▌ ██ ▓██▒ ██ ▀█ █ ██▒ ▀█▒▓█ ▀ ▒██▒ ██▒ ██ ▀█ █
|
| 13 |
+
▒██ ▀█▄ ▒██▒ ░██ █▌▓██ ▒██░▓██ ▀█ ██▒▒██░▄▄▄░▒███ ▒██░ ██▒▓██ ▀█ ██▒
|
| 14 |
+
░██▄▄▄▄██ ░██░ ░▓█▄ ▌▓▓█ ░██░▓██▒ ▐▌██▒░▓█ ██▓▒▓█ ▄ ▒██ ██░▓██▒ ▐▌██▒
|
| 15 |
+
▓█ ▓██▒░██░ ░▒████▓ ▒▒█████▓ ▒██░ ▓██░░▒▓███▀▒░▒████▒░ ████▓▒░▒██░ ▓██░
|
| 16 |
+
▒▒ ▓▒█░░▓ ▒▒▓ ▒ ░▒▓▒ ▒ ▒ ░ ▒░ ▒ ▒ ░▒ ▒ ░░ ▒░ ░░ ▒░▒░▒░ ░ ▒░ ▒ ▒
|
| 17 |
+
▒ ▒▒ ░ ▒ ░ ░ ▒ ▒ ░░▒░ ░ ░ ░ ░░ ░ ▒░ ░ ░ ░ ░ ░ ░ ▒ ▒░ ░ ░░ ░ ▒░
|
| 18 |
+
░ ▒ ▒ ░ ░ ░ ░ ░░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ▒ ░ ░ ░
|
| 19 |
+
░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░
|
| 20 |
+
```
|
| 21 |
+
### This repository contains quantized conversions of the AI Dungeon 2 checkpoint, "model_v5".
|
| 22 |
+
*For use with frontends that support GGML quantized GPT-2 models.*
|
| 23 |
+
|
| 24 |
+
*Last updated on 2023-09-23.*
|
| 25 |
+
|
| 26 |
+
Model | RAM usage (KoboldCpp) | RAM usage (Oobabooga)
|
| 27 |
+
:--:|:--:|:--:
|
| 28 |
+
aid2classic-ggml-q4_0.bin | 984.1 MiB | 1.4 GiB
|
| 29 |
+
aid2classic-ggml-q4_1.bin | 1.1 GiB | 1.5 GiB
|
| 30 |
+
aid2classic-ggml-q5_0.bin | 1.2 GiB | 1.6 GiB
|
| 31 |
+
aid2classic-ggml-q5_1.bin | 1.2 GiB | 1.7 GiB
|
| 32 |
+
aid2classic-ggml-q8_0.bin | 1.7 GiB | 2.2 GiB
|
| 33 |
+
aid2classic-ggml-f16.bin | 3.2 GiB | 3.6 GiB
|
| 34 |
+
|
| 35 |
+
**Notes:**
|
| 36 |
+
- KoboldCpp [[bfc696f]](https://github.com/LostRuins/koboldcpp/tree/bfc696fcc452975dbe8967c39301ba856d04a030) was tested without OpenBLAS.
|
| 37 |
+
- Oobabooga [[895ec9d]](https://github.com/oobabooga/text-generation-webui/tree/895ec9dadb96120e8202a83052bf9032ca3245ae) was tested with with the `--model <model> --loader ctransformers --model_type gpt2` launch arguments.
|
| 38 |
+
- ggerganov/ggml [[8ca2c19]](https://github.com/ggerganov/ggml/tree/8ca2c19a3bb8622954d858fbf6383522684eaf34)'s gpt-2 conversion script was used for conversion and quantization.
|
| 39 |
+
- The original model was found in the `generator/gpt2/models/model_v5` directory of [AI Dungeon 2 Unleashed](https://henk.tech/aid/).
|