Osaurus AI

⚠️ Tool calling — parser notice. This model emits Qwen-XML tool calls (<function=name><parameter=...> inside <tool_call>), not Hermes JSON. This bundle declares tool_parser: "qwen" in its config — use a runtime whose tool parser accepts the Qwen XML dialect. Hermes-only parsers will fail on tool turns (dropped/mangled calls). Runtime support for the correct parser is rolling out; until your runtime has it, expect tool-use errors.

OsaurusAI/Qwen3.8-Flash-Next-JANG_1L

The smallest tier — the full 181B-parameter system in ~60 GiB (or ~41 GiB resident with the n-gram table served from SSD).

A JANG bundle of Qwen/Qwen3.8-Flash-Next — the Qwen4-architecture preview: a 125B mixture-of-experts (512 experts, 6B active) with a 51B hashed n-gram embedding, Gated DeltaNet + Qwen Sparse Attention hybrid layers, gated-residual streams, and vision+video towers — quantized for Apple Silicon / MLX. Text, image and video weights are all present in this exact bundle. MTP head omitted (floor tier).

Runtime support is landing — this repo is private while the serving path is finalized.

Quality (measured, 5,931 held-out positions vs bf16)

JANG ladder

Tier Size RAM w/ SSD-table median KL top-1 top-5 top-10
JANG_1L 59.8 GiB ~41 GiB 0.0362 86.7% 97.5% 98.8%
JANG_2L 65.3 GiB ~48 GiB 0.0260 88.2% 98.2% 99.0%
JANG_4S 71.8 GiB ~53 GiB 0.0161 89.4% 98.7% 99.4%
JANG_4M 96.0 GiB ~73 GiB 0.0042 94.4% 99.7% 99.9%
JANG_6S 106.3 GiB ~83 GiB 0.0035 94.7% 99.7% 99.9%

Margin-conditioned flip curves are monotone-decreasing on every tier — quantization noise lives in the reference model's own uncertainty band, with zero disagreement at high-confidence positions on the upper tiers.

The n-gram table & memory

The 51B n-gram embedding can be served directly from SSD by supporting runtimes (16 row-reads per token); the "RAM w/ SSD-table" column above is the resident footprint in that mode. SSD-table caching is fixed and runs at full decode speed on current runtime builds.

What's in the bundle

  • Vision + video: the full vision tower and both image and video preprocessors ship in this exact bundle — image-text-to-text and video understanding work out of the box on supporting runtimes (image and video token ids, mRoPE positions, and the merger are all present).
  • Multi-token prediction: not included in this tier (smallest-footprint build). Use JANG_2L or above for speculative decode.
  • Thinking + agentic: thinking mode on by default with three reasoning efforts and preserved thinking history; Hermes-style tool calling; the instruct preset gives direct non-thinking responses.
  • Long context: 262,144 tokens native, extensible to 1M with YaRN.

Serving contract

  • Thinking mode ON by default: temperature=1.0, top_p=0.95, top_k=20
  • Instruct mode: temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5
  • Reasoning efforts low / medium / xhigh (default xhigh) and preserve_thinking (default on) via chat-template kwargs
  • Context 262,144 native, extensible to 1M with YaRN
  • EOS [248046, 248044] · tool calls: Hermes-style <tool_call>

Quantized and validated by Jinho Jangeric@osaurus.ai

Downloads last month
81
Safetensors
Model size
20B params
Tensor type
F16
·
U32
·
BF16
·
I64
·
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for OsaurusAI/Qwen3.8-Flash-Next-JANG_1L

Finetuned
(24)
this model