Quant Details
EXL2 quant of Undi's merge using Bluemoon-Light's Nemo parquet. Cache 8bit seems more stable than 4bit in my testing with Nemo based models.
https://huggingface.co/Undi95/Lumimaid-Magnum-12B
https://huggingface.co/datasets/ParasiticRogue/Bluemoon-Light
Original Model Card Information
Merge of Lumimaid and Magnum as requested by some.
I used the new DELLA merge method in mergekit and added a finetune of Nemo only on Claude input, trained on 16k ctx, in the mix.
Prompt template: Mistral
<s>[INST] {input} [/INST] {output}</s>
- Downloads last month
- 146
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.