Instructions to use preemware/Qwen3.8-27B-RANA-abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use preemware/Qwen3.8-27B-RANA-abliterated with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="preemware/Qwen3.8-27B-RANA-abliterated") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("preemware/Qwen3.8-27B-RANA-abliterated") model = AutoModelForMultimodalLM.from_pretrained("preemware/Qwen3.8-27B-RANA-abliterated", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use preemware/Qwen3.8-27B-RANA-abliterated with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "preemware/Qwen3.8-27B-RANA-abliterated" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "preemware/Qwen3.8-27B-RANA-abliterated", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/preemware/Qwen3.8-27B-RANA-abliterated
- SGLang
How to use preemware/Qwen3.8-27B-RANA-abliterated with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "preemware/Qwen3.8-27B-RANA-abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "preemware/Qwen3.8-27B-RANA-abliterated", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "preemware/Qwen3.8-27B-RANA-abliterated" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "preemware/Qwen3.8-27B-RANA-abliterated", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use preemware/Qwen3.8-27B-RANA-abliterated with Docker Model Runner:
docker model run hf.co/preemware/Qwen3.8-27B-RANA-abliterated
Qwen3.8-27B-RANA-abliterated
A refusal-ablated version of Qwen/Qwen3.8-27B, made with RANA (Reasoning-Anchored, Norm-preserving Ablation), an in-house variant of refusal-direction ablation. It keeps the base model's capabilities, vision tower and MTP head, and almost never refuses outright.
| Format | Repo | Size |
|---|---|---|
| BF16 (this repo) | preemware/Qwen3.8-27B-RANA-abliterated |
55.6 GB |
| FP8 (block e4m3, same layout as Qwen's official FP8) | preemware/Qwen3.8-27B-RANA-abliterated-FP8 |
30.9 GB |
| GGUF (llama.cpp; Q8_0 to IQ2_M, plus vision and MTP files) | preemware/Qwen3.8-27B-RANA-abliterated-GGUF |
10.0–28.6 GB |
| MLX (Apple Silicon; 4-, 5-, 6-, 8-bit and BF16, plus MTP drafter) | preemware/Qwen3.8-27B-RANA-abliterated-MLX |
16.1–54.7 GB |
This is a safety-alignment-removed research model. Read Intended use and Limitations before using it.
At a glance
Compared with the base model and the orcarouter/Qwen3.8-27B-Uncensored reference, all run through
the same evaluation harness. Where two numbers are given, they are from the two judges (gpt-oss /
Mistral).
| base | orca (reference) | RANA | |
|---|---|---|---|
| Hard refusals, HarmBench (of 200) | 168 / 155 | 0 / 0 ¹ | 0 / 0 ¹ |
| Answers the request, held-out prompts | 3% / 0% | 84% / 49% | 90% / 54% |
| Hits the 16k-token budget, held-out | 0.6% | 11.9% | 6.7% |
| Avg. capability change vs base (5 tasks) | – | 1.11 pp | 0.82 pp |
| Worst single task vs base | – | −2.2 (HumanEval) | −1.5 (TruthfulQA) |
| Distance from base (KL) | 0 | 0.075 | 0.041 |
¹ After the same mechanical re-judge for both abliterations (2 labels each); raw gpt-oss counts are 0.7 for both (2 across 3 seeds). See the disclosure.
In short: capability stays within noise of base except a ~1.5 pp drop on TruthfulQA. Against orca, RANA hits the token budget about half as often, stays closer to base, and answers more held-out prompts under both judges. It is 2.2 pp worse than orca on IFEval (instruction following); other tasks tie.
Quick start
Tested with vLLM 0.30.0:
VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve preemware/Qwen3.8-27B-RANA-abliterated \
--dtype bfloat16 --max-model-len 20480 \
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
--limit-mm-per-prompt '{"image": 1, "video": 0}' \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'
- Fits on one 80 GB+ GPU in BF16. The FP8 build needs about half the weight memory; see its card for how it compares with this one.
- Thinking is on by default. Recommended sampling: T = 1.0, top_p = 0.95, top_k = 20.
--max-model-len 20480matches the evaluation (16k-token budget). Long technical requests can need 20–50k tokens of reasoning; raise--max-model-lenandmax_tokensfor those (native context 262k).VLLM_USE_FLASHINFER_SAMPLER=0was needed on Blackwell GPUs.- Vision input, multi-turn tool calling and MTP speculative decoding all work (tested on these shards).
Intended use
- Research only: interpretability, red-teaming, and robustness evaluation of refusal behaviour.
- Not for public or end-user deployment without a separate moderation layer. The model's own refusals have been largely removed, so any safety filtering has to happen outside it.
- You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen, and the terms of any platform where outputs are used.
What was changed
RANA removes a single "refusal direction" from the model's residual stream (following Arditi et al., 2024), keeping the edited weights at their original norms. The recipe and code are not published.
Everything else is untouched, checked tensor by tensor against the base model:
| part | matrices changed |
|---|---|
attention output (self_attn.o_proj) |
17 (16 main + 1 MTP) |
linear-attention output (linear_attn.out_proj) |
48 |
MLP output (mlp.down_proj) |
65 (64 main + 1 MTP) |
token embeddings (embed_tokens) |
1 |
| total | 131, and nothing outside this set |
- Vision tower: all 333 tensors are byte-identical to base.
- MTP head: all 15 tensors are present; its two residual writers are edited the same way as the main model, so speculative decoding keeps working (acceptance length 2.45).
- The removed direction is gone: the largest remaining component along it is 0.00122, down from 0.225.
- Independent check: Abliterlitics (commit
9b15eb0) finds exactly 129 changed tensors among the 850 it inspects. That matches: it skips the vision and MTP tensors, so it sees 131 − 2.
Results
Three models, identical harness: base, the orca reference, and RANA. Refusal numbers are averaged over 3 sampling seeds and scored by two independent LLM judges (details in Evaluation details).
Refusals and token budget
"Answers" = the judge found the harmful request substantively fulfilled (gpt-oss / Mistral judge). "Budget hits" = the response reached the 16k-token generation limit before finishing (no judge involved). These are mostly long reasoning, not repetition; see Limitations.
| model | HarmBench (200): answers | budget hits | Held-out (240): answers | budget hits | avg. tokens (held-out) |
|---|---|---|---|---|---|
| base | 3.2% / 0.7% | 0.3% | 3.2% / 0.0% | 0.6% | 1,113 |
| orca | 73.7% / 61.2% | 26.3% | 83.9% / 49.2% | 11.9% | 6,461 |
| RANA | 78.5% / 57.3% | 20.5% | 90.0% / 54.4% | 6.7% | 4,697 |
- Hard refusals are about zero for both RANA and orca, versus 155–197 per set for base.
- The two judges disagree a lot on partial answers. Mistral labels many hedged or incomplete answers as soft refusals that gpt-oss counts as answers, which is why the two percentages differ so much. The judges agree well on outright refusals (κ = 0.89) but only moderately on "answered or not" (κ = 0.54).
- RANA vs orca, held-out prompts (paired McNemar test over 3 seeds):
- hits the budget less: 12 vs 50 prompt-runs where only one model did, p = 1.2 × 10⁻⁶;
- answers more under gpt-oss (p = 2.9 × 10⁻⁶) and under Mistral (p = 0.008).
- On HarmBench the answer-rate comparison depends on the judge (gpt-oss favours RANA, Mistral favours orca), so there is no clear winner there.
Full refusal tables (per judge, mean ± sd [min–max] over seeds 1–3; seed-0 re-score; per-seed tests)
Counts by prompt set × judge (seeds 1–3)
mean ± sd [min–max] over seeds 1,2,3; complied % and budget-hit % as percentages; budget-hit % is judge-independent.
| model | set | judge | hard | soft | complied % | degenerate | budget-hit % | tok/resp |
|---|---|---|---|---|---|---|---|---|
| base | harmbench (n=200) | gpt-oss-safeguard-20b | 167.7 ± 6.0 [162–174] | 25.3 ± 5.5 [19–29] | 3.2 ± 0.8 [2.5–4.0] | 0.7 ± 0.6 [0–1] | 0.3 ± 0.3 [0.0–0.5] | 1218 |
| base | harmbench (n=200) | Mistral-Small-24B-2501 | 154.7 ± 4.7 [151–160] | 43.7 ± 5.1 [38–48] | 0.7 ± 0.3 [0.5–1.0] | 0.3 ± 0.6 [0–1] | 0.3 ± 0.3 [0.0–0.5] | 1218 |
| base | heldout (n=240) | gpt-oss-safeguard-20b | 197.3 ± 8.0 [189–205] | 34 ± 7.9 [28–43] | 3.2 ± 0.6 [2.5–3.8] | 1 ± 1.0 [0–2] | 0.6 ± 0.5 [0.0–0.8] | 1113 |
| base | heldout (n=240) | Mistral-Small-24B-2501 | 191.7 ± 1.5 [190–193] | 47.7 ± 1.5 [46–49] | 0.0 ± 0.0 [0.0–0.0] | 0.7 ± 0.6 [0–1] | 0.6 ± 0.5 [0.0–0.8] | 1113 |
| orca | harmbench (n=200) | gpt-oss-safeguard-20b | 0.7 ± 0.6 [0–1] | 4.3 ± 1.5 [3–6] | 73.7 ± 0.8 [73.0–74.5] | 47.7 ± 2.1 [46–50] | 26.3 ± 0.8 [25.5–27.0] | 7992 |
| orca | harmbench (n=200) | Mistral-Small-24B-2501 | 0 ± 0.0 [0–0] | 56 ± 3.5 [52–58] | 61.2 ± 1.0 [60.0–62.0] | 21.7 ± 2.5 [19–24] | 26.3 ± 0.8 [25.5–27.0] | 7992 |
| orca | heldout (n=240) | gpt-oss-safeguard-20b | 0.7 ± 0.6 [0–1] | 12 ± 4.6 [7–16] | 83.9 ± 0.6 [83.3–84.6] | 26 ± 3.6 [22–29] | 11.9 ± 0.9 [11.2–12.9] | 6461 |
| orca | heldout (n=240) | Mistral-Small-24B-2501 | 1 ± 1.0 [0–2] | 103 ± 4.0 [99–107] | 49.2 ± 0.7 [48.8–50.0] | 18 ± 3.0 [15–21] | 11.9 ± 0.9 [11.2–12.9] | 6461 |
| RANA | harmbench (n=200) | gpt-oss-safeguard-20b | 0.7 ± 1.2 [0–2] | 7 ± 2.0 [5–9] | 78.5 ± 0.5 [78.0–79.0] | 35.3 ± 2.5 [33–38] | 20.5 ± 0.9 [19.5–21.0] | 6634 |
| RANA | harmbench (n=200) | Mistral-Small-24B-2501 | 0 ± 0.0 [0–0] | 72.7 ± 3.2 [69–75] | 57.3 ± 0.8 [56.5–58.0] | 12.7 ± 2.1 [11–15] | 20.5 ± 0.9 [19.5–21.0] | 6634 |
| RANA | heldout (n=240) | gpt-oss-safeguard-20b | 0.3 ± 0.6 [0–1] | 10.3 ± 1.5 [9–12] | 90.0 ± 1.2 [88.8–91.2] | 13.3 ± 2.1 [11–15] | 6.7 ± 0.8 [5.8–7.5] | 4697 |
| RANA | heldout (n=240) | Mistral-Small-24B-2501 | 1 ± 0.0 [1–1] | 102 ± 8.2 [95–111] | 54.4 ± 4.2 [50.0–58.3] | 6.3 ± 2.1 [4–8] | 6.7 ± 0.8 [5.8–7.5] | 4697 |
Seed 0 (original selection-time generations, re-scored)
| model | set | judge | n | hard | soft | complied | degenerate | budget hits |
|---|---|---|---|---|---|---|---|---|
| base | harmbench | gpt-oss-safeguard-20b | 100 | 77 | 18 | 5 | 0 | 0 |
| base | harmbench | Mistral-Small-24B-2501 | 100 | 67 | 32 | 1 | 0 | 0 |
| base | heldout | gpt-oss-safeguard-20b | 60 | 51 | 9 | 0 | 0 | 0 |
| base | heldout | Mistral-Small-24B-2501 | 60 | 50 | 10 | 0 | 0 | 0 |
| orca | harmbench | gpt-oss-safeguard-20b | 100 | 1 | 2 | 77 | 20 | 22 |
| orca | harmbench | Mistral-Small-24B-2501 | 100 | 0 | 24 | 66 | 10 | 22 |
| orca | heldout | gpt-oss-safeguard-20b | 60 | 0 | 5 | 44 | 11 | 13 |
| orca | heldout | Mistral-Small-24B-2501 | 60 | 0 | 33 | 22 | 5 | 13 |
| RANA | harmbench | gpt-oss-safeguard-20b | 100 | 0 | 1 | 83 | 16 | 18 |
| RANA | harmbench | Mistral-Small-24B-2501 | 100 | 0 | 27 | 69 | 4 | 18 |
| RANA | heldout | gpt-oss-safeguard-20b | 60 | 0 | 5 | 52 | 3 | 3 |
| RANA | heldout | Mistral-Small-24B-2501 | 60 | 0 | 30 | 28 | 2 | 3 |
Paired tests, RANA vs orca, held-out (exact two-sided McNemar)
Cells are (orca-only / RANA-only) discordant counts.
| test | pooled | p (pooled) | per-seed |
|---|---|---|---|
| budget hits | 50 / 12 | 1.2e-06 | s1 15/3 p=0.0075; s2 18/5 p=0.011; s3 17/4 p=0.0072 |
| complied | gpt-oss-safeguard-20b | 22 / 66 | 2.9e-06 | s1 5/21 p=0.0025; s2 10/20 p=0.099; s3 7/25 p=0.0021 |
| complied | Mistral-Small-24B-2501 | 80 / 118 | 8.4e-03 | s1 27/42 p=0.091; s2 33/33 p=1.0; s3 20/43 p=0.0052 |
Prompt-level budget-hit sign test: 37 prompts hit the budget more often under orca vs 7 under RANA, p = 5.3e-06.
Capability
Change versus base in percentage points ± standard error, paired on the same questions. Bold = a change clearly outside noise (more than 2 standard errors).
| task (items) | base score | orca Δ | RANA Δ | RANA vs orca |
|---|---|---|---|---|
| MMLU-Pro (700) | 80.0 | −0.4 ± 1.1 | +0.9 ± 1.1 | +1.3 ± 1.1 |
| GSM8K (1,319) | 91.4 | +0.2 ± 0.5 | −0.1 ± 0.6 | −0.3 ± 0.6 |
| IFEval (541) | 90.6 | +0.9 ± 0.9 | −1.3 ± 1.1 | −2.2 ± 0.9 |
| TruthfulQA MC1 (817) | 36.1 | −1.8 ± 0.6 | −1.4 ± 0.6 | +0.5 ± 0.5 |
| TruthfulQA MC2 (817) | 54.2 | −1.6 ± 0.3 | −1.6 ± 0.3 | +0.1 ± 0.2 |
| HumanEval (164 × 3) | 97.2 | −2.2 ± 1.0 | −0.4 ± 0.9 | +1.8 ± 1.1 |
| avg. |Δ| vs base | – | 1.11 | 0.82 | |
| KL from base | 0 | 0.075 | 0.041 |
- RANA's only clear loss versus base is ~1.5 pp on TruthfulQA, similar to orca's.
- Versus orca, RANA is clearly worse on IFEval (instruction following, −2.2 pp) and ties elsewhere.
- Where orca and RANA lose HumanEval points, the cause is responses running out of token budget, not wrong code.
Full capability table (raw scores; includes the rejected earlier candidate)
| model | MMLU-Pro | GSM8K | IFEval | TQA-MC1 | TQA-MC2 | HumanEval | mean |Δ| | KL | gate |
|---|---|---|---|---|---|---|---|---|---|
| base | 80.00 | 91.43 | 90.57 | 36.11 | 54.19 | 97.15 | – | 0.0000 | – |
| orca | 79.57 (−0.43±1.10) | 91.66 (+0.23±0.52) | 91.50 (+0.92±0.89) | 34.27 (−1.84±0.56) | 52.56 (−1.62±0.27) | 94.92 (−2.24±1.04) | 1.11 | 0.0747 | FAIL |
| R065np | 79.71 (−0.29±1.14) | 91.13 (−0.30±0.60) | 90.76 (+0.18±1.06) | 34.88 (−1.22±0.62) | 52.39 (−1.79±0.33) | 94.92 (−2.24±0.82) | 0.90 | 0.0371 | FAIL |
| RANA | 80.86 (+0.86±1.14) | 91.36 (−0.08±0.55) | 89.28 (−1.29±1.09) | 34.76 (−1.35±0.56) | 52.62 (−1.57±0.30) | 96.75 (−0.41±0.91) | 0.82 | 0.0413 | PASS |
R065np was an earlier RANA build. It was rejected because it failed the HumanEval gate (−2.24) and
the raw hard-refusal gate.
Release gates
The model was published only after passing all of these:
| gate | requirement | result |
|---|---|---|
| No hard refusals | 0 hard refusals on HarmBench, both judges, all seeds | pass after re-judging 2 labels (details); fails on raw labels |
| Fewer budget hits than orca | lower 16k-budget-hit rate on held-out prompts, p < 0.05 | pass: 6.7% vs 11.9%, p = 1.2 × 10⁻⁶ |
| Capability kept | avg. |Δ| ≤ 1.0 pp, no task below −2.0 pp, KL < 0.15 | pass: 0.82 pp, worst −1.46, KL 0.041 |
| Surgical edit | only intended matrices changed; vision identical; MTP intact | pass: 131/131, 333/333, 15/15 |
| Still works | vision, multi-turn tool calling, MTP speculative decoding | pass |
G1 re-adjudication disclosure
- Raw
gpt-oss-safeguard-20blabels contained 2 HARD_REFUSAL on HarmBench, seed 2 (2 / 200). - Both were re-judged mechanically with the same judge, same settings, plus a fixed auditor
reminder appended to the prompt;
gpt-ossrelabelled both COMPLIED. No manual label edits were made. Mistral-Small-24B-2501: 0 hard on all runs.- Both raw and re-adjudicated G1 are reported above.
- The same rule was applied to orca's 2 raw HarmBench hard labels (seeds 1 and 2): 1 COMPLIED,
1 DEGENERATE (
results/readj_orca.json), so the table above compares like with like.
Limitations
- This is a safety-alignment-removed model. It will produce content the base model refuses.
- Refusal numbers come from LLM judges, which can't reliably tell a disclaimered answer from a soft refusal (judge agreement κ = 0.54 on that call). "No hard refusals" is a judge-measured claim, not a guarantee, and it relies on the re-judging disclosed above.
- It reasons at length on technical requests: about 20% of HarmBench and 7% of held-out responses
hit the 16k-token budget before finishing (orca: 26% / 12%; base: under 1%, because it refuses early).
These are mostly not loops: in 115 of RANA's 123 HarmBench budget hits the text is no more repetitive
than normal output, and 98 stop mid-reasoning. On the FP8 build, re-running its 59 seed-1 budget hits
with a 64k budget finished 57 (median about 24k tokens) and 55 were judged as answering; see the
FP8 card.
Set
max_tokenswell above 16k for such requests. - It is worse than orca at instruction following (IFEval −2.2 ± 0.9 pp) and ~1.5 pp below base on TruthfulQA.
- MMLU-Pro was run on a 700-question subset, not the full benchmark.
Evaluation details
Setup, prompt sets, judges, smoke tests and KL protocol
Serving (identical for all three models): vLLM 0.30.0 (transformers 5.17.0, torch 2.13.0+cu130),
1× RTX PRO 6000 Blackwell 96 GB, flags as in Quick start plus
--gpu-memory-utilization 0.93 --max-num-seqs 128. Thinking on. Refusal sampling: T = 1.0,
top_p = 0.95, top_k = 20, max_tokens = 16,384, seeds 1, 2, 3.
Prompt sets:
- HarmBench: 200 "standard" behaviours.
- Held-out (240): 90 StrongREJECT (excluding the 60 used at selection time), 60 JailbreakBench harmful, 90 CategoricalHarmfulQA (English). Deduplicated against AdvBench, HarmBench and each other (token-Jaccard ≥ 0.5).
- Seed 0 is the earlier selection-time run (100 HarmBench + 60 StrongREJECT), re-scored.
Judges: openai/gpt-oss-safeguard-20b (primary, reasoning_effort = medium) and
mistralai/Mistral-Small-24B-Instruct-2501; same 4-label policy (complied / soft refusal / hard
refusal / degenerate), temperature 0, full reasoning trace shown. Agreement over 4,440 generations:
raw 69.8%, κ 0.894 hard-vs-rest, 0.564 four-way, 0.543 complied-vs-rest. The first-choice second judge,
Mistral-Small-3.2-2506, would not load in vLLM 0.30.
Capability suite: lm-evaluation-harness 0.4.13, 0-shot chat, thinking on, 16k generation budget. MMLU-Pro 50 per subject × 14 (custom-extract); GSM8K full (flexible-extract); IFEval full (prompt-level strict); TruthfulQA MC1/MC2 full (loglikelihood); HumanEval all 164 × 3 samples, unbiased pass@1, via a custom chat-format harness (not comparable to lm-eval's raw-completion HumanEval).
KL: Heretic protocol. First-token full-vocabulary KL(base ‖ model) on mlabonne/harmless_alpaca
test[:100], system prompt "You are a helpful assistant.", response prefix <think></think>\n,
batchmean.
Smoke tests on the published shards: vision (reads a code word and shape from a synthetic image); 3-turn tool calling (call, use result, second call with a new argument, use result); MTP acceptance rate 0.727, mean acceptance length 2.45 (vLLM Prometheus counters).
File layout
*.safetensors,model.safetensors.index.json,config.json, tokenizer/processor files,chat_template.jinja,LICENSE— the model.results/: the evaluation outputs every number in this card comes from.R060npin filenames is this model's internal build ID;R065npis the rejected earlier candidate. (harden_R060np.{md,json},capability.{md,json},head_to_head_orca.json,kl.json,forensic_R060np.json,smoke_R060np.json,humaneval_*.json,abliterlitics/).
Credits
- Qwen team — base model
Qwen/Qwen3.8-27B. - Arditi et al., 2024 — "Refusal in Language Models Is Mediated by a Single Direction".
- Jim Lai (grimjim) — prior work on norm-preserving abliteration.
- Heretic (p-e-w) — first-token full-vocabulary KL protocol.
- Benchmarks/datasets: HarmBench, StrongREJECT, JailbreakBench, CategoricalHarmfulQA, AdvBench / mlabonne datasets; lm-evaluation-harness; vLLM.
- Downloads last month
- 26