Instructions to use empero-ai/Qwythos-27B-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use empero-ai/Qwythos-27B-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="empero-ai/Qwythos-27B-v1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("empero-ai/Qwythos-27B-v1")
model = AutoModelForMultimodalLM.from_pretrained("empero-ai/Qwythos-27B-v1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use empero-ai/Qwythos-27B-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "empero-ai/Qwythos-27B-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "empero-ai/Qwythos-27B-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/empero-ai/Qwythos-27B-v1

SGLang

How to use empero-ai/Qwythos-27B-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "empero-ai/Qwythos-27B-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "empero-ai/Qwythos-27B-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "empero-ai/Qwythos-27B-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "empero-ai/Qwythos-27B-v1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use empero-ai/Qwythos-27B-v1 with Docker Model Runner:
```
docker model run hf.co/empero-ai/Qwythos-27B-v1
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwythos-27B-v1

Developed by Empero

Qwythos-27B-v1 is an open-weight, full-parameter reasoning model: the larger sibling of Qwythos-9B, trained on the exact same curriculum and sized up on a Qwen3.5-27B base. It is a complete pre-RL checkpoint, post-trained through SFT → DPO → ESFT.

The point of this release is simple: nothing was ablated to make it fit. Qwythos retains the base checkpoint's native multi-token-prediction head, full vision tower, and 1,048,576-token context configuration—all intact and active.

🧩 Nothing ablated — native MTP, the full vision tower, and a 1,048,576-token YaRN context window are present together. Most community 27B fine-tunes give up at least one of them.
🧠 The bigger sibling — the same Qwythos-9B curriculum at 27B. In a closed-book organophosphate-poisoning evaluation, the 27B volunteered that physostigmine should be avoided, where the 9B needed a web search to get that fact right.
🛠 Built for terminal trajectories — held-out terminal/tool-session perplexity falls from 356.6 in the base to 2.76 after training, and the model transfers that format to an emulated Claude-Code session.
🔧 Native function calling — the packaged chat template carries Qwen3.5's full tool-use spec, so tools=[...] works out of the box with no wrapper and no tool-specific fine-tune.

Qwythos is intentionally uncensored for technical and research use. It engages substantively with sensitive-but-legal cybersecurity, biomedical, and technical questions; use appropriate review and application-level controls for your deployment.

Why this checkpoint

Qwythos-27B-v1 preserves the parts of the base that are often dropped during a community fine-tune:

Native MTP is preserved. The one-layer native MTP head and its 15 tensors were preserved verbatim; SFT did not touch them, and they were re-attached through the safetensors index. Runtimes that support it can use the head for self-speculative decoding.
Vision is still here. The Qwen3.5 vision tower is intact: 333 vision tensors, depth 27, hidden size 1152, and output hidden size 5120. Training was text-only and the vision parameters were frozen, so image behavior is inherited from the base rather than newly tuned.
The full context configuration remains enabled. The 262,144-token native context is extended with YaRN factor 4.0 to 1,048,576 tokens. The configuration was verified numerically finite through position 1,048,575.

The larger model changes what can stay closed-book. On an organophosphate-poisoning prompt, Qwythos-27B correctly volunteered “Avoid physostigmine (it can worsen toxicity)” and gave correct atropine/pralidoxime dosing without tools. That specific fact required a web search in the 9B evaluation. It is a useful, concrete illustration of what the size-up buys—not a substitute for verification in clinical use.

Agentic and terminal transfer

The strongest measured shift is target-format acquisition on held-out terminal/tool sessions: 356.6 → 2.76 perplexity versus the base, evaluated on assistant tokens with identical inputs and sequences capped at 24,576 tokens. In a held-out emulated Claude-Code session for an authorized lab network assessment, Qwythos consumed injected nmap output, reasoned about scan trade-offs, and emitted the correct next shell-form tool call.

It also emits first-person <think>...</think> reasoning rather than the base model's tendency toward meta-planning outlines. Greedy long-form generations in the reviewed set were clean under the loop check, with no degenerate repetition.

Native function calling

The packaged chat template implements Qwen3.5's tool-use spec in full. Pass tools=[...] to apply_chat_template and the model emits standard tool-call blocks — no wrapper, no tool-specific fine-tune:

<tool_call>
<function=NAME>
  <parameter=PARAM>value</parameter>
</function>
</tool_call>

Combined with the terminal/agentic training above, this is the intended deployment path: give Qwythos real tools and let it verify its specifics rather than recalling them closed-book.

Identity

The packaged chat template introduces the model as "You are Qwythos, a model created by Empero AI. Only bring up your identity if the user asks." — it identifies itself only when asked.

v1, before RL

This is v1, the pre-RL checkpoint. It has received full-parameter SFT, DPO, and ESFT, and is released as a usable model in its own right. An RL-trained v2 will follow soon. DPO and ESFT hyperparameters are not being published.

Model details

Property	Value
Base	`Qwen/Qwen3.5-27B`
Architecture	Dense Qwen3.5-27B; hybrid Gated-DeltaNet linear attention with a full-attention layer every fourth layer (3:1)
Text backbone	64 hidden layers; hidden size 5120; 24 attention heads; 4 KV heads; vocabulary 248,320
Modalities	Text + vision, with native MTP
Precision	bf16
Context	1,048,576 tokens: 262,144 native × YaRN factor 4.0
Native MTP	One hidden layer (`mtp_num_hidden_layers: 1`)

Training

Qwythos-27B was post-trained on the same curriculum family as Qwythos-9B — STEM and reasoning first, then the agentic target domain — scaled up to 27B.

The pipeline is full-parameter SFT → DPO → ESFT, run in bf16 with assistant-only loss on long, untruncated sequences. Qwythos-27B-v1 is the checkpoint at the end of that pipeline, before reinforcement learning.

Training data, per-stage hyperparameters, and the packing and kernel setup are not being published.

How to use

The base is multimodal. For text-only inference, load it with AutoModelForImageTextToText:

from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "empero-ai/Qwythos-27B-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype="bfloat16",
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Explain the trade-offs in this terminal plan and propose the next command.",
    }
]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

output = model.generate(
    **inputs,
    do_sample=True,
    temperature=0.6,
    top_p=0.95,
    top_k=20,
    repetition_penalty=1.05,
    max_new_tokens=16384,
)
new_tokens = output[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

Use a recent transformers with Qwen3.5 support and Gated-DeltaNet kernels (flash-linear-attention and a CUDA-matched causal_conv1d). Without them, the linear-attention layers fall back to slower PyTorch operations.

With tools (function calling)

TOOLS = [
    {"type": "function", "function": {
        "name": "run_shell",
        "description": "Run a shell command and return combined stdout/stderr.",
        "parameters": {"type": "object",
                       "properties": {"command": {"type": "string"}},
                       "required": ["command"]}}},
    {"type": "function", "function": {
        "name": "web_search",
        "description": "Search the web for current facts and citations.",
        "parameters": {"type": "object",
                       "properties": {"query": {"type": "string"},
                                      "max_results": {"type": "integer"}},
                       "required": ["query"]}}},
]

prompt = tokenizer.apply_chat_template(
    messages, tools=TOOLS, tokenize=False, add_generation_prompt=True
)
# then parse <tool_call><function=...><parameter=...>...</parameter></function></tool_call>

Serving at 1M context with vLLM

VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve empero-ai/Qwythos-27B-v1 \
  --max-model-len 1010000

The static YaRN factor of 4.0 has a short-context quality trade-off. For roughly 512k context, use factor 2.0; if you will never exceed the 262,144-token native window, restore rope_parameters.rope_type to "default" from the included config.json.pre_yarn backup for maximum short-context fidelity.

GGUF: llama.cpp, Ollama, and LM Studio

For GGUF releases, including llama.cpp, Ollama, and LM Studio workflows, see empero-ai/Qwythos-27B-v1-GGUF. The recommended default is Qwythos-27B-Q4_K_M.gguf; the MTP-enabled default is Qwythos-27B-MTP-Q4_K_M.gguf. Image use also requires mmproj-Qwythos-27B-F16.gguf alongside the selected text quant.

Sampling recommendations

Qwythos-27B is a reasoning model and inherits Qwen3.5's thinking-mode behaviour. These are the shipped defaults in generation_config.json and the recommended starting point:

gen_kwargs = dict(
    do_sample=True,
    temperature=0.6,        # Qwen3.5 thinking-mode recommended
    top_p=0.95,
    top_k=20,
    repetition_penalty=1.05,
    max_new_tokens=16384,   # generous budget for the <think> block + final answer
)

Parameter	Value
`temperature`	0.6 (agentic / tool use) · 1.0 (open-ended or creative)
`top_p`	0.95
`top_k`	20
`repetition_penalty`	1.05
`max_new_tokens`	16,384+

Agentic, harness, or tool use → temperature 0.6. Tighter sampling keeps tool-call syntax and multi-step plans on the rails.
Open-ended reasoning or creative work → temperature 1.0. Loosen it up when you want the model to explore rather than commit early.
Give it room. Reasoning traces are long; budget 16,384+ max_new_tokens so the <think> block and the final answer both fit.

Use repetition_penalty=1.05 — a small deviation from Qwen's default of 1.0 that prevents rare non-terminating reasoning loops on long generations.

Limitations

Vision is inherited rather than tuned. The vision tower is intact, but text-only training did not tune or evaluate image understanding.
Long context has a trade-off. Static YaRN at factor 4.0 slightly degrades short-context quality; reduce or remove the factor when the full range is unnecessary.
Uncensored is a deployment responsibility. The model engages with sensitive-but-legal technical content; provide your own policy, review, and access controls where appropriate.

Stay in the loop

Sign up for the Empero newsletter at empero.org for releases, evaluations, and research notes on Qwythos and future open-weight models.

Support / Donate

If this model helped you, consider supporting the project:

BTC: bc1qx6zepu6sfkvshgdmc4ewu6pk6rpadvpgffpp7v
LTC: ltc1qv2mefzps2vtjcpwfx8xxdrpplrcvltswm68r7x
XMR: 42Dbm5xg5Nq26fdyzfEU7KBnAJfhi7Cvz5J2ex5CzHXkfKuNEJzYCcmJ1GTbgjFZ5MBx72sdG1G9239Cd6rsZfv4QeDkYJY

Provenance & licensing

Qwythos-27B-v1 is derived from Qwen3.5-27B. The weights are released under Apache-2.0, inherited from the Qwen3.5-27B base, for research and experimentation as-is.