Instructions to use OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M

A Korean-first JANG_4M MLX bundle of Kakao's Kanana-2 30B-A3B Mixture-of-Experts model, tuned to run fast on Apple Silicon via Osaurus.

Converted from kakaocorp/kanana-2-30b-a3b-instruct-2601.

Why this model

Kanana-2 is one of the strongest open Korean LLMs — Kakao trained it with a tokenizer ~30% more efficient on Korean and a 6-language mix (Korean, English, Japanese, Chinese, Thai, Vietnamese). It is a 30.7B-parameter MoE that activates only ~3B parameters per token (top-6 of 128 routed experts + 2 shared), so you get large-model Korean quality at small-model decode speed. JANG_4M compresses it to ~17 GB so it fits comfortably in unified memory on a MacBook.

Bundle

Field	Value
Source	`kakaocorp/kanana-2-30b-a3b-instruct-2601`
Architecture	`deepseek_v3` / `DeepseekV3ForCausalLM` (MLA attention)
Total / active params	30.7B / ~3B per token
Decoder layers	48 (layer 0 dense, 1–47 MoE)
Hidden size	2048
Experts	128 routed, top-6, + 2 shared (`moe_intermediate_size` 768)
Attention	MLA — `kv_lora_rank` 512, no query LoRA, `qk_nope` 128 / `qk_rope` 64, `v_head_dim` 128
Context	32,768 tokens
Weight format	`jang_affine` (JANG v2, instant mmap load)
Tier bits	attention 8-bit · router/gate fp16 · MoE expert + MLP bulk 4-bit · embed/lm_head bookends high-precision
Average bits/weight	4.26
Shards	32 safetensors
Bundle size	~17 GB
Capabilities	`text` (family `deepseek_v3`, reasoning `qwen3`, tools `qwen`, cache `mla`)

Quantization policy (JANG_4M)

JANG_4M is a mixed-precision affine scheme designed for MoE: the precision-sensitive pieces stay wide while the redundant expert bulk is compressed.

8-bit affine: self-attention projections (MLA q/kv compression + decompression, o_proj).
fp16 passthrough: MoE router/gate weights (routing is extremely precision-sensitive), all RMSNorms.
4-bit affine: routed-expert and dense-MLP gate/up/down projections (the bulk of the weights).
High-precision bookends: token embeddings and lm_head.

The per-tensor bit widths are recoverable directly from the weight/scale shapes; jang_config.json is authoritative for the JANG runtime.

Modalities

Modality	Status
Text	supported
Vision / Audio / Video	not present in source — text-only model

Languages

Korean (primary), English, Japanese, Chinese, Thai, Vietnamese.

Reasoning & tools

The source chat template is ChatML-style (<|im_start|>role) with <think>…</think> reasoning and <tool_call>{json}</tool_call> tool calls. The bundle stamps reasoning_parser: qwen3, tool_parser: qwen. Thinking is off by default for the instruct variant; enable it through your runtime's thinking toggle.

Runtime

Built for the Osaurus / vMLX Apple-Silicon runtime. Loading requires a deepseek_v3-aware MLX loader that honors jang_config.json and the bundle's mixed per-tensor bit widths. tokenizer_config.json, generation_config.json, and chat_template.jinja are preserved from the source.

Notes

This is a freshly converted bundle. The base model's published Korean benchmarks (e.g. KMMLU) reflect the unquantized weights; JANG_4M-level evaluation for this bundle is pending and will be added here when complete.
MTP: the source ships an MTP layer; speculative decoding is not enabled in this bundle (mtp: none for autoregressive decode).

Credits

Base model: Kakao Corp. — Kanana-2, released under the Kanana License.
JANG conversion & Osaurus packaging: Jinho Jang — eric@osaurus.ai

Downloads last month: 38

Safetensors

Model size

33B params

Tensor type

U32

F16

MLX

Hardware compatibility

Quantized

Model tree for OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M

Base model

kakaocorp/kanana-2-30b-a3b-base-2601

Finetuned

kakaocorp/kanana-2-30b-a3b-mid-2601

Finetuned

kakaocorp/kanana-2-30b-a3b-instruct-2601

Finetuned

(1)

this model