Hugging Face's logo

FluffyKaeloky
/
Midnight-Miqu-103B-v1.5-exl2-4.0bpw-rpcal

Text Generation
Transformers
Safetensors
llama
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
4-bit precision
exl2
Model card Files Community
1
Edit model card
MidnightMiqu

Midnight-Miqu-103B-v1.5-exl2-4.0bpw-rpcal

This is a 4.0bpw EXL2 quant of FluffyKaeloky/Midnight-Miqu-103B-v1.5

The pippa file used for calibration is optimised for roleplay. The measurement file can be found in the files if you want to do your own quants.

Details about the model and the merge info can be found at the fp16 model link above.

Downloads last month
15
Inference Examples
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.