Instructions to use SubMaroon/Kanimus-26B-A4B-FFT-heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SubMaroon/Kanimus-26B-A4B-FFT-heretic with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="SubMaroon/Kanimus-26B-A4B-FFT-heretic")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("SubMaroon/Kanimus-26B-A4B-FFT-heretic")
model = AutoModelForMultimodalLM.from_pretrained("SubMaroon/Kanimus-26B-A4B-FFT-heretic", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SubMaroon/Kanimus-26B-A4B-FFT-heretic with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SubMaroon/Kanimus-26B-A4B-FFT-heretic"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SubMaroon/Kanimus-26B-A4B-FFT-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/SubMaroon/Kanimus-26B-A4B-FFT-heretic

SGLang

How to use SubMaroon/Kanimus-26B-A4B-FFT-heretic with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SubMaroon/Kanimus-26B-A4B-FFT-heretic" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SubMaroon/Kanimus-26B-A4B-FFT-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SubMaroon/Kanimus-26B-A4B-FFT-heretic" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SubMaroon/Kanimus-26B-A4B-FFT-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use SubMaroon/Kanimus-26B-A4B-FFT-heretic with Docker Model Runner:
```
docker model run hf.co/SubMaroon/Kanimus-26B-A4B-FFT-heretic
```

Kanimus-26B-A4B-FFT-heretic (v4 QK-LoRA Merge)

An experimental Dark Roleplay model. Two modifications were applied on top of an abliterated Animus base: a task-arithmetic injection into the attention Query and Key projections from a Claude Opus distillation, and a stylistic LoRA baked into the weights.

This is a single test run and not suitable for release. See Evaluation and Known Issues before using it.

Base

Vortex5/Gemma-4-26B-A4B-Animus-V14.1-FFT-heretic, which is an abliterated variant of Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT. Refusal behaviour therefore differs from stock Gemma 4 before any of the changes below were applied.

Everything outside the modified tensors comes from that base unchanged: the MoE router, all 128 routed experts, embeddings, layer norms, and the vision tower.

Architecture notes

Gemma 4 26B A4B has 30 text layers. Layers 5, 11, 17, 23 and 29 use full attention; the remaining 25 use sliding attention with a 1024 window.

The full-attention layers set attention_k_eq_v: true and have no v_proj. Keys are reused as values there. The model has 115 attention projections in total: 30 q_proj, 30 k_proj, 25 v_proj, 30 o_proj.

Projection shapes differ between the two layer types:

	q_proj	k_proj	v_proj	o_proj
sliding	2816 → 4096	2816 → 2048	2816 → 2048	4096 → 2816
full	2816 → 8192	2816 → 1024	absent	8192 → 2816

Step 1: QK injection (task arithmetic)

Applied only to q_proj and k_proj. Formula:

W_out = W_animus + α * (W_opus − W_parent)

Reference models:

Target: TeichAI/gemma-4-26B-A4B-it-Claude-Opus-Distill-v2
Parent: unsloth/gemma-4-26B-A4B-it

The unsloth checkpoint is used as the parent because the Opus distillation was trained from it. Using google/gemma-4-26B-A4B-it instead would fold the repackaging difference between the two into the delta.

Four separate α values were used:

α_sliding_q = 0.50
α_sliding_k = 0.50
α_global_q  = 0.50
α_global_k  = 0.25

α_global_k is halved because on full-attention layers k_proj also serves as the value matrix, so a delta there is effectively a KV injection.

q_norm and k_norm were not transferred and remain from the base. This is a hybrid, not a transfer of the complete QK block.

Arithmetic was done in FP32, output saved in BF16.

Merging was done with a hand-written safetensors script. mergekit does not currently handle the Gemma 4 checkpoint layout.

Post-merge invariant. After Step 1 and before the LoRA bake, every tensor outside the 60 q/k tensors was verified bitwise equal to the base with torch.equal. This does not describe the released weights, which include the LoRA bake described below.

Step 2: LoRA bake

SubMaroon/Dark-Goetia-26B-A4B-LoRA-v4 was merged into the Step 1 weights.

Adapter configuration:

r = 32, alpha = 64
Targets: q_proj, k_proj, v_proj, o_proj and the shared dense MLP (mlp.gate_proj, mlp.up_proj, mlp.down_proj) on all 30 layers, 205 modules total
Routed experts, router, embeddings, lm_head, vision tower: not targeted

Bake:

for module in peft_model.modules():
    if isinstance(module, LoraLayer):
        module.set_scale("default", 0.20)
merged = peft_model.merge_and_unload(safe_merge=True)

set_scale multiplies the training scaling, which is alpha / r = 2. The effective delta added to the base is therefore 0.40 * BA, not 0.20 * BA.

Caveat on the scale. LoRA v4 was not trained on this merge. The 0.40 figure was carried over from earlier work on a different base and has not been recalibrated for the abliterated Animus base used here.

What was not changed

No lm_head swap was performed. tie_word_embeddings remains true and there is no separate lm_head tensor.

The vision tower is carried over from the base unchanged and untested. Image input has not been exercised in any of the testing described below.

tokenizer_config.json was patched: extra_special_tokens was converted from a list to a dict. The list form is present in the upstream base and breaks convert_hf_to_gguf.py on current transformers.

Evaluation

None. No quantitative evaluation was run on this model.

Specifically, the following were planned and skipped: paired ΔNLL against the base on held-out roleplay text, MoE routing load diagnostics, and long-context state tracking at 16k / 32k / 64k. There are no numbers to report for any of them.

What was done: a handful of manual generations in English and Russian, compared by eye against the base model on partly matching prompts. That is the entire basis for anything said about behaviour here.

Known issues

Referent drift (English, unmeasured). In test generations a character's established name was sometimes replaced mid-scene by a similar-sounding invented name. Agent of an action and narrative person also swapped within a scene in some samples. This was not observed in base-model output on comparable prompts. Cause not yet identified; the QK injection is the current suspect.

Token-level artifacts in Russian. Latin-script fragments appear inside Russian words, along with malformed and non-existent words. The same artifacts appear in the base model at a similar rate, so this is inherited rather than introduced. Whether it comes from the abliteration, from Q4_K_M quantization or from both has not been isolated.