Not-For-All-Audiences
nsfw
| license: cc-by-nc-4.0 | |
| tags: | |
| - not-for-all-audiences | |
| - nsfw | |
| ## Exl2 version of [Undi95/SolarMaid-v0.1.1](https://huggingface.co/Undi95/SolarMaid-v0.1.1) | |
| ## branch | |
| 3bh8 : 3bpw h8 | |
| 3.5bh8 : 3.5bpw h8 | |
| 3.8bh8 : 3.8bpw h8 | |
| 4bh8 : 4bpw h8 | |
| 6bh8 : 6bpw h8 | |
| 8bh8 : 8bpw h8 | |
| Using ThePile [0007.parquet](https://huggingface.co/datasets/EleutherAI/the_pile_deduplicated/resolve/refs%2Fconvert%2Fparquet/default/train/0007.parquet) as dataset | |
| Quantization settings : ```python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp -cf SolarMaid-v0.1.1-8bpw-h8-exl2 -c 0007.parquet -l 8192 -b 8 -hb 8 -ml 8192``` | |
| ```python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp2 -cf SolarMaid-v0.1.1-6bpw-h8-exl2 -c 0007.parquet -l 8192 -b 6 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192``` | |
| ```python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp3 -cf SolarMaid-v0.1.1-4bpw-h8-exl2 -c 0007.parquet -l 8192 -b 4 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192``` | |
| ```python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp4 -cf SolarMaid-v0.1.1-3bpw-h8-exl2 -c 0007.parquet -l 8192 -b 3.8 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192``` | |
| ```python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp5 -cf SolarMaid-v0.1.1-2.4bpw-h8-exl2 -c 0007.parquet -l 8192 -b 3.5 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192``` | |
| ```python convert.py -i models/Undi95_SolarMaid-v0.1.1 -o SolarMaid-v0.1.1-temp5 -cf SolarMaid-v0.1.1-2.4bpw-h8-exl2 -c 0007.parquet -l 8192 -b 3 -hb 8 -m SolarMaid-v0.1.1-temp/measurement.json -ml 8192``` | |
| ### below this line is original readme | |
| SolarMaid is a model made primarely for ERP (Erotic Roleplay) without being too dumb, it use 40 out of 48 layers of "SOLAR Instruct 10.7B", and 32 out of 32 layers of "Noromaid 7B v0.1.1". | |
| Despite being a little unstable, [benchmark stay high](https://huggingface.co/Undi95/SolarMaid-v0.1.1/blob/main/SolarMaid-v0.1.1.json), and can output some hot sh*t. | |
| I'm working on making it more stable. | |
| <!-- description start --> | |
| ## Description | |
| This repo contains fp16 files of SolarMaid v0.1.1. | |
| <!-- description end --> | |
| <!-- description start --> | |
| ## Models used | |
| - upstage/SOLAR-10.7B-Instruct-v1.0 | |
| - NeverSleep/Noromaid-7b-v0.1.1 | |
| <!-- description end --> | |
| <!-- prompt-template start --> | |
| ## Prompt template: Alpaca | |
| ``` | |
| Below is an instruction that describes a task. Write a response that appropriately completes the request. | |
| ### Instruction: | |
| {prompt} | |
| ### Response: | |
| ``` | |
| ## Prompt template: Custom (SOLAR) | |
| ``` | |
| ### User: | |
| {prompt} | |
| ### Assistant: | |
| ``` | |
| If you want to support me, you can [here](https://ko-fi.com/undiai). |