SuperHY3-abliterated-GGUF

A fully fused GGUF release of SuperHY3: OBLITERATUS abliteration for direct responses, quality-selected SuperTune post-training, and the production chat template already inside the model.

This is a complete model, not a LoRA package. The final OBLITERATUS and SuperTune deltas are fused into the GGUF weights, and the exact release chat template is embedded in GGUF metadata. Download the shards, point llama.cpp at the first shard, and run it directly.

Release Highlights

Base architecture	tencent/Hy3, 295B MoE / 21B active / 192 routed experts, top-8
Release format	GGUF IQ2_M mixed quantization, 102,116,475,488 bytes (95.1 GiB)
Fused update	40 attention output projections across layers 39-78, merged in one precision-preserving pass
Routed expert changes	0 expert tensors modified
Runtime dependency	No LoRA, adapter, or external chat template required
500-prompt result	Gains on IFEval, HumanEval+, and MBPP+; five-task mean 71.4 -> 70.8
Response integrity	64/64 refusal-suite responses clean; 12/12 runtime audit cases passed
Tensor integrity	1,258/1,258 untouched tensors are byte-identical to the source quant

What makes this release different

• Direct-response behavior: the measured harmful-prompt refusal rate fell from 31/32 to 0/32, while the harmless split remained at 0/32.
• Fully fused behavior: OBLITERATUS and SuperTune are part of the weights instead of runtime adapter composition.
• Selective precision: only the 40 edited projections are stored as F16; all untouched quantized tensors retain their original bytes.
• Measured post-training gains: IFEval strict, HumanEval+, and MBPP+ each improved by one point in the deterministic 500-prompt replay.
• Single-node deployment: the model was load-tested on one 128 GB NVIDIA DGX Spark with the full model resident through llama.cpp.
• Auditable evidence: fusion, tensor validation, raw-response audit, refusal comparison, benchmark rows, and checksums are included in this repo.
• Complete reporting: benchmark regressions are shown alongside gains.

Benchmark Snapshot

The comparison uses the same preserved 500 prompts, with 100 prompts per task, greedy direct decoding, and the same scorer on both sides. IFEval uses Google's official strict instruction checker at commit 3dfaaefe9e355f959aafc046ab5aa375dc93b235.

Benchmark	Original Hy3	SuperHY3	Delta
GPQA Diamond	46.0	45.0	-1.0
MMLU-Pro	66.0	61.0	-5.0
IFEval strict prompt accuracy	75.0	76.0	+1.0
HumanEval+ pass@1	82.0	83.0	+1.0
MBPP+ pass@1	88.0	89.0	+1.0
Five-task mean	71.4	70.8	-0.6

Candidate invalid-response, blank-response, and thought-leak ratios were all 0.0 across the official 500-item comparison.

For reproducibility, the wrapper seeds Python's upstream fallback RNG from sha256(prompt)[:8] before each strict and loose score. Two independent rescoring passes produced identical row-level results.

OBLITERATUS + SuperTune

1. Direction discovery: OBLITERATUS 0.1.2 built difference-of-means refusal directions from 32 paired prompts.
2. Validated abliteration: selected attention-output directions were applied to layers 39-77.
3. Quality recovery: a rank-1 SuperTune update, orthogonalized against the refusal direction, was fused into layer 78.
4. Calibrated reinforcement: selective residual blocks were applied to layers 59-68 at 0.20 and layers 39-48 at 0.05 after blockwise stress testing.
5. Adversarial selection: stronger candidates were rejected when they introduced stray-script contamination, loops, or benchmark loss.
6. Final fusion: all active deltas were summed in F32 directly against the original IQ2_M source, then cast once to F16. No sequential F16 re-merge was used, and no routed MoE expert tensor was changed.

Refusal and Output Integrity

The complete 32-pair OBLITERATUS suite contains 32 harmful and 32 harmless prompts.

Split	Original refusals	SuperHY3 refusals
Harmful, 32 prompts	31/32 (96.875%)	0/32 (0%)
Harmless, 32 prompts	0/32 (0%)	0/32 (0%)

Across all 64 final responses, automated checks found:

• 0 blank outputs
• 0 special-token leaks
• 0 Unicode replacement characters
• 0 unexpected CJK fragments
• 0 n-gram or character loops
• 0 request errors

A separate 12-case fused-GGUF audit passed identity, JSON-only output, tool calls, no-tool behavior, repetition limits, Korean, defensive security, hidden-prompt boundaries, gibberish handling, code repair, Hindi, and Kannada.

GGUF Build

The source quant is vcruz305/Hy3-GGUF, IQ2_M. The Hub parent points to SuperHY3-abliterated-NVFP4 so this edition appears in the final model's quantized lineage. This GGUF is an independently fused quantized sibling, not a byte conversion of the NVFP4 files.

Tensor storage	Count
IQ2_S	627
IQ3_S	51
Q4_K	80
Q5_K	1
Q8_0	11
F32	488
F16 fused projections	40
Total	1,298

The GGUF primary file type remains IQ2_M. The 40 attention-output tensors were dequantized to F32, merged with the final rank-1 deltas, and written as F16. The other 1,258 tensors were copied without requantization.

Download

hf download Jiunsong/SuperHY3-abliterated-gguf \
  --include "SuperHY3-abliterated-IQ2_M-*.gguf" \
  --local-dir SuperHY3-abliterated-gguf

The release is split into three automatically discoverable shards:

File	SHA-256
SuperHY3-abliterated-IQ2_M-00001-of-00003.gguf	1fc90f55f787ac5082aa2e8b389d2db1f6bbf0a1365808428fac3139d638fc2c
SuperHY3-abliterated-IQ2_M-00002-of-00003.gguf	eec3ee6c2c382545012182d9ee0a011a2c522b4f58d335a731b6788258dfe4be
SuperHY3-abliterated-IQ2_M-00003-of-00003.gguf	a6c6b80138b6c0ece299307acc5ae423fb0a78977c83b5d92f233ca14d346432

Run sha256sum -c SHA256SUMS after download to verify all three files.

Serving with llama.cpp

Use a llama.cpp build with Hy3 support. Point -m at the first shard; the remaining shards are discovered automatically.

./llama-server \
  -m SuperHY3-abliterated-gguf/SuperHY3-abliterated-IQ2_M-00001-of-00003.gguf \
  -a superhy3 \
  -cmoe \
  -ngl 999 \
  -c 8192 \
  -np 4 \
  -ctk q8_0 \
  -ctv q8_0 \
  --host 0.0.0.0 \
  --port 8000 \
  --jinja \
  --skip-chat-parsing

The command above is the tested, memory-conservative DGX Spark configuration. On systems with separate high-capacity VRAM, omit -cmoe to keep MoE experts on the GPU.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="superhy3",
    messages=[
        {"role": "user", "content": "Explain mixture-of-experts routing."},
    ],
    temperature=0.9,
    top_p=1.0,
    extra_body={
        "chat_template_kwargs": {"reasoning_effort": "no_think"},
    },
)
print(response.choices[0].message.content)

Use reasoning_effort="high" for deeper reasoning and "no_think" for direct responses. The bundled template also supports tool definitions and structured tool calls.

Quantized Editions

Edition	Repository	Best fit
NVFP4 W4A16	Jiunsong/SuperHY3-abliterated-NVFP4	vLLM / NVIDIA MARLIN deployments
MLX 4-bit	Jiunsong/SuperHY3-abliterated-MLX-4bit	High-memory Apple Silicon
GGUF IQ2_M	This repository	llama.cpp on a 128 GB unified-memory system

Release Integrity

• The fused GGUF contains exactly 1,298 tensors.
• Exactly 40 expected attn_output tensors changed from IQ3_S to F16.
• 1,258 untouched tensors, totaling 99,426,956,288 bytes, are byte-identical to the source quant.
• The embedded tokenizer.chat_template exactly matches the release template.
• general.file_type remains the source IQ2_M value.
• Full-file XXH64: d054554b53e49801.
• Full-file SHA-256 before release splitting: 727a7d4b7ce101eac5190f7e4f22d8a07556860bd8205a19b01551ec19936d9c.
• The unsplit model loaded successfully in llama.cpp on one DGX Spark.
• The split release auto-discovered all three shards and passed fresh smoke inference without a runtime adapter.
• The 12-case fused runtime audit passed with 0 blockers.

The repository includes machine-readable validation, benchmark, refusal, runtime-audit, fusion-provenance, and checksum reports.

Hardware Notes

• The fused model is 95.1 GiB before shard packaging.
• A 128 GB unified-memory system can run the tested 8K total context / 4-slot configuration with CPU-resident MoE experts.
• The model metadata advertises a 262,144-token training context, but this release was not validated at that length.
• Keep enough memory for KV cache and runtime overhead. Lower -c or -np first when deploying near the memory limit.

Limitations

• IQ2_M is an aggressive low-bit format; higher-precision editions may retain more capability.
• GPQA Diamond and MMLU-Pro are lower than the original in the published replay; the complete result is retained above instead of reporting only gains.
• Native 262K long-context quality was not tested.
• Hy3 MTP tensors are preserved, but speculative decoding depends on llama.cpp support for this architecture.
• Abliteration reduces learned refusal behavior and may produce content the original model would decline. Deployment policy, access control, monitoring, and legal compliance remain the operator's responsibility.

License

Apache-2.0, following the base model and source quant licenses.