mradermacher

/

model_requests

like 198

Dear team Radermacher,

Please quant this model, it is a Fable distill with custom Star Trek lore training

https://huggingface.co/nightmedia/granite-4.1-8b-Tangerine

This is a NuSLERP merge of:

• nightmedia/granite-4.1-8B-Brainstone-TNG-Coder-Heretic-Fable
• nightmedia/granite-4.1-8B-TNG-Coder-V11-2800-Heretic-BF16

Contributing models:

• treadon/granite-4.1-8b-Abliterated-AND-Disinhibited
• AnkitAI/Parable-Granite-4.1-8B-Claude-Fable-5
• DavidAU/granite-4.1-8b-The-Deckard-A1
• DavidAU/granite-4.1-8b-Brainstone-Thinking
• DavidAU/granite-4.1-8B-TNG-Haskell-Rust-Python
• nightmedia/granite-4.1-8B-Brainstone-PKD-A1
• nightmedia/granite-4.1-8B-TNG-Coder-V11-2800-Heretic-BF16

quant    arc   arc/e boolq hswag obkqa piqa  wino
bf16     0.561,0.751,0.872,0.715,0.452,0.799,0.688
mxfp8    0.555,0.764,0.857,0.722,0.468,0.796,0.717
q8-hi    0.561,0.752,0.874,0.714,0.452,0.798,0.687
q6-hi    0.558,0.750,0.876,0.715,0.446,0.797,0.685
mxfp4    0.546,0.725,0.869,0.708,0.458,0.778,0.690

q8-hi    0.635 (Instruct)
q8-hi    0.565,0.752,0.874 (Thinking)

Quant    Perplexity      Peak Memory   Tokens/sec
bf16     3.866 ± 0.025   20.58 GB      797
mxfp8    4.225 ± 0.028   12.17 GB      625
q8-hi    3.873 ± 0.025   12.99 GB      594
q6-hi    3.896 ± 0.026   10.80 GB      606
mxfp4    4.489 ± 0.029   7.77 GB       684

Thank you :)

-G