|
--- |
|
license: cc-by-nc-4.0 |
|
tags: |
|
- roleplay |
|
- llama3 |
|
- sillytavern |
|
--- |
|
|
|
> [!IMPORTANT] |
|
> **Updated!** <br> |
|
> Version (**v2**) files added! With imatrix data generated from the FP16 and conversions directly from the BF16. <br> |
|
> This is a more disk and compute intensive so lets hope we get GPU inference support for BF16 models in llama.cpp. <br> |
|
> Hopefully avoiding any losses in the model conversion, as has been the recently discussed topic on Llama-3 and GGUF lately. <br> |
|
> If you are able to test them and notice any issues let me know in the discussions. |
|
|
|
> [!IMPORTANT] |
|
> **Relevant:** <br> |
|
> These quants have been done after the fixes from [**llama.cpp/pull/6920**](https://github.com/ggerganov/llama.cpp/pull/6920) have been merged. <br> |
|
> Use **KoboldCpp** version **1.64** or higher, make sure you're up-to-date. |
|
|
|
> [!TIP] |
|
> I apologize for disrupting your experience. <br> |
|
> My upload speeds have been cooked and unstable lately. <br> |
|
> If you **want** and you are **able to**... <br> |
|
> You can [**support my various endeavors here (Ko-fi)**](https://ko-fi.com/Lewdiculous). <br> |
|
|
|
GGUF-IQ-Imatrix quants for [NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS). |
|
|
|
**Author:** <br> |
|
"This model received the Orthogonal Activation Steering treatment, **meaning it will rarely refuse any request.**" |
|
|
|
> [!WARNING] |
|
> Compatible SillyTavern presets [here (simple)](https://huggingface.co/ChaoticNeutrals/Poppy_Porpoise-v0.7-L3-8B/tree/main/Official%20Poppy%20Porpoise%20ST%20Presets) or [here (Virt's Roleplay Presets - recommended)](https://huggingface.co/Virt-io/SillyTavern-Presets). <br> |
|
> Use the latest version of KoboldCpp. **Use the provided presets for testing.** <br> |
|
> Feedback and support for the Authors is always welcome. <br> |
|
> If there are any issues or questions let me know. |
|
|
|
> [!NOTE] |
|
> For **8GB VRAM** GPUs, I recommend the **Q4_K_M-imat** (4.89 BPW) quant for up to 12288 context sizes. |
|
|
|
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/JUxfdTot7v7LTdIGYyzYM.png) |
|
|
|
**Original model information:** |
|
|
|
## Lumimaid 0.1 |
|
|
|
<center><div style="width: 100%;"> |
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/630dfb008df86f1e5becadc3/d3QMaxy3peFTpSlWdWF-k.png" style="display: block; margin: auto;"> |
|
</div></center> |
|
|
|
This model uses the Llama3 **prompting format** |
|
|
|
Llama3 trained on our RP datasets, we tried to have a balance between the ERP and the RP, not too horny, but just enough. |
|
|
|
We also added some non-RP dataset, making the model less dumb overall. It should look like a 40%/60% ratio for Non-RP/RP+ERP data. |
|
|
|
This model includes the new Luminae dataset from Ikari. |
|
|
|
This model have received the Orthogonal Activation Steering treatment, meaning it will rarely refuse any request. |
|
|
|
If you consider trying this model please give us some feedback either on the Community tab on hf or on our [Discord Server](https://discord.gg/MtCVRWTZXY). |
|
|
|
## Credits: |
|
- Undi |
|
- IkariDev |
|
|
|
## Description |
|
|
|
This repo contains FP16 files of Lumimaid-8B-v0.1-OAS. |
|
|
|
Switch: [8B](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1) - [70B](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-70B-v0.1) - [70B-alt](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-70B-v0.1-alt) - [8B-OAS](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS) - [70B-OAS](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-70B-v0.1-OAS) |
|
|
|
## Training data used: |
|
- [Aesir datasets](https://huggingface.co/MinervaAI) |
|
- [NoRobots](https://huggingface.co/datasets/Doctor-Shotgun/no-robots-sharegpt) |
|
- [limarp](https://huggingface.co/datasets/lemonilia/LimaRP) - 8k ctx |
|
- [toxic-dpo-v0.1-sharegpt](https://huggingface.co/datasets/Undi95/toxic-dpo-v0.1-sharegpt) |
|
- [ToxicQAFinal](https://huggingface.co/datasets/NobodyExistsOnTheInternet/ToxicQAFinal) |
|
- Luminae-i1 (70B/70B-alt) (i2 was not existing when the 70b started training) | Luminae-i2 (8B) (this one gave better results on the 8b) - Ikari's Dataset |
|
- [Squish42/bluemoon-fandom-1-1-rp-cleaned](https://huggingface.co/datasets/Squish42/bluemoon-fandom-1-1-rp-cleaned) - 50% (randomly) |
|
- [NobodyExistsOnTheInternet/PIPPAsharegptv2test](https://huggingface.co/datasets/NobodyExistsOnTheInternet/PIPPAsharegptv2test) - 5% (randomly) |
|
- [cgato/SlimOrcaDedupCleaned](https://huggingface.co/datasets/cgato/SlimOrcaDedupCleaned) - 5% (randomly) |
|
- Airoboros (reduced) |
|
- [Capybara](https://huggingface.co/datasets/Undi95/Capybara-ShareGPT/) (reduced) |
|
|
|
|
|
## Models used (only for 8B) |
|
|
|
- Initial LumiMaid 8B Finetune |
|
- Undi95/Llama-3-Unholy-8B-e4 |
|
- Undi95/Llama-3-LewdPlay-8B |
|
|
|
## Prompt template: Llama3 |
|
|
|
``` |
|
<|begin_of_text|><|start_header_id|>system<|end_header_id|> |
|
|
|
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|> |
|
|
|
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
|
|
|
{output}<|eot_id|> |
|
``` |
|
|
|
## Others |
|
|
|
Undi: If you want to support us, you can [here](https://ko-fi.com/undiai). |
|
|
|
IkariDev: Visit my [retro/neocities style website](https://ikaridevgit.github.io/) please kek |