Instructions to use KRAFTON/Raon-1-nano with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KRAFTON/Raon-1-nano with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KRAFTON/Raon-1-nano", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("KRAFTON/Raon-1-nano", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KRAFTON/Raon-1-nano with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KRAFTON/Raon-1-nano"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KRAFTON/Raon-1-nano",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/KRAFTON/Raon-1-nano

SGLang

How to use KRAFTON/Raon-1-nano with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KRAFTON/Raon-1-nano" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KRAFTON/Raon-1-nano",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KRAFTON/Raon-1-nano" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KRAFTON/Raon-1-nano",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use KRAFTON/Raon-1-nano with Docker Model Runner:
```
docker model run hf.co/KRAFTON/Raon-1-nano
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Raon-1 nano

Raon-1 nano is a dense language model with 1.5B effective parameters (3.1B total, including a 1.5B per-layer embedding table), developed by KRAFTON AI. It is designed to follow instructions, maintain conversational context, and call the right tools in both Korean and English. As part of the Raon open model family, it is released under the Apache 2.0 license.

Highlights

Faithful, natural conversation: Leads the comparison models on the instruction-following, context-following, and Korean conversation benchmarks shown below.
Korean and English: Trained on Korean and English, with solid English capabilities alongside its Korean strengths.
Tool calling: Tool calling within conversations, for everyday tasks such as office work and mobile control.
On-device deployment: Available in GGUF, MLX, and LiteRT-LM formats for local inference, including on Apple silicon.
Long context: Natively supports up to 262,144 tokens of context.

Raon-1 nano vs. open models of similar size

Model Overview

Raon-1 nano uses a hybrid dense architecture, consisting of 18 linear attention layers and 6 full attention layers.

Raon Delta Attention (RDA): Gated delta-rule linear attention with separate erase and write gates and a fixed-size recurrent state.
Raon-mHC: Raon-mHC reduces the computational cost of DeepSeek's manifold-constrained hyper-connections (mHC), which speeds up training.
Per-layer embedding: Each layer has its own small 256-dimensional embedding for every token. After its MLP, the layer uses it to adjust its output before passing it on.
Hybrid layout: 18 of the 24 layers are Raon Delta Attention layers with a fixed-size state. Only the 6 MLA layers keep a KV cache, about 7 KiB per token in BF16.
Tied embeddings: The LM head shares the input embedding.

Field	Value
Architecture	Dense
Total Parameters	3.1B (including the 1.5B per-layer embedding table)
Layers	24
Hidden Size	1,536
Attention	Hybrid (RDA 3 : Gated MLA 1)
Position Encoding	NoPE (no positional encoding)
Number of Attention Heads	16 for RDA; 32 for Gated MLA
Attention Head Dimension	128 for RDA; 192 QK / 128 V, for Gated MLA
Query / KV LoRA Rank	512 / 512, for Gated MLA
FFN Intermediate Size	6,144
Activation Function	SwiGLU
Residual Connection	Raon-mHC w/ 4 streams
Per-Layer Embedding	256 per layer
Vocabulary Size	259,840
LM Head	Tied
MTP	1 layer, 55M
Context Length	262,144 tokens

The checkpoint on HuggingFace contains 3.2B parameters: 3.1B for the main model and 55M for the Multi-Token Prediction (MTP) module.

Training

MoE-to-Dense: Raon-1 nano was derived from Raon-1 mini preview (30B-A4B) through MoE-to-Dense pruning and knowledge distillation.
Continued pre-training: 3T tokens.
Post-training: supervised fine-tuning (SFT) and on-policy distillation (OPD) from Raon-1 mini preview, followed by reinforcement learning (RL).

Benchmark Results

Red indicates that Raon-1 nano is the best among the compared models; bold indicates the best model when another model leads.

Category	Benchmark	Raon-1 nano (E1.5B)	Qwen3.5 2B	Gemma 4 E2B	MiniCPM5 2B	LFM3 1.7B
Total params (effective)		3.1B (E1.5B)	2.3B	5.1B (E2.3B)	2.5B	1.7B
Knowledge	MMLU-Redux	81.1	81.5	78.9	84.0	81.1
	KMMLU-Redux (ko)	50.9	46.1	44.9	43.0	45.2
	HAE-RAE 1.1 (ko)	44.2	36.2	45.2	36.1	34.3
	CLIcK (ko)	65.2	54.8	62.2	54.6	54.1
Instruction Following	IFBench	75.9	45.7	39.4	63.8	73.7
Instruction Following	MultiChallenge	40.5	37.0	32.3	28.1	39.0
Context Following	FACTS Grounding	44.4	13.8	40.5	28.4	7.6
	PersonaMem v2¹	38.8	29.2	32.7	29.3	28.8
	PrefEval²	44.0	26.1	23.3	24.4	19.6
Tool Use	Office Work Bench (ko, internal)³ Workplace	49.2	15.7	19.8	33.9	39.8
	BFCL v4 General function calling	56.3	41.2	37.9	61.0	67.2
	AndroidControl⁴ Mobile	36.0	26.9	35.0	31.7	17.5
	NL2SH-ALFA Terminal	58.0	43.9	62.6	61.4	57.3
	BrowseComp Plus Search	34.5	25.7	17.2	37.2	33.6
Korean Conversation	Chatbot Korean (ko, internal)⁵	91.0	42.2	87.6	41.6	58.3

¹ PersonaMem v2: 32k context setting.
² PrefEval: generation task, average of 10 / 70 / 150 conversation turns.
³ Office Work Bench: an internal benchmark measuring diverse office tasks performed in Slack, including scheduling meetings and searching documents or threads. Not publicly released.
⁴ AndroidControl: average of high-level (HL) and low-level (LL) instruction settings.
⁵ Chatbot Korean: an internal benchmark measuring Korean colloquial ability in chatbot conversations across diverse situations. Not publicly released.
Raon-1 nano sampling: temperature 0.7 for Knowledge, temperature 1.0 for Tool Use, temperature 0.1 with presence penalty 1.5 for the others. Baselines use the sampling parameters recommended in each model card.

Sampling parameters

Raon-1 nano is designed for thinking mode; disabling it is not recommended.

Recommended settings:

Task	temperature	presence_penalty
General (chat, knowledge Q&A, instruction following, etc.)	0.1	1.5
Agentic / code	1.0	0.0

Thinking is enabled by default in the chat template. generation_config.json defines the EOS and padding token IDs; it does not set sampling parameters. The output token limit (max_new_tokens, max_tokens) includes both thinking and the final answer; increase it for longer responses.

Quickstart

Raon-1 nano supports inference with Transformers, vLLM, llama.cpp, MLX, and LiteRT-LM. The examples below show how to run the model locally or launch an OpenAI-compatible server with thinking mode enabled (the LiteRT-LM bundle answers without thinking). The vLLM and llama.cpp examples use the recommended general sampling settings above; the Transformers example uses greedy decoding for a simple first run.

Transformers

Use Transformers to load Raon-1 nano and generate responses directly in Python. The Transformers example uses Python 3.12, PyTorch 2.13.0 with CUDA 13.0, Transformers 5.16.1, and Accelerate 1.15.0.

Create an environment and install a PyTorch build suitable for your hardware, then install:

uv venv --python 3.12 --seed
source .venv/bin/activate

# Install the appropriate PyTorch build in this environment first.
pip install \
    "transformers==5.16.1" \
    "accelerate==1.15.0"

Then run the model:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "KRAFTON/Raon-1-nano"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "user", "content": "Tell me about KRAFTON AI."}
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    enable_thinking=True,
    truncate_history_thinking=False,
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    do_sample=False,
)

output_ids = outputs[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(output_ids, skip_special_tokens=False).strip())

The example prints both thinking and the final answer, retaining the </think> separator.

The implementation uses plain PyTorch and requires no vLLM installation. RDA processes prompt tokens sequentially, so long-prompt prefill can be slow.

vLLM

Use vLLM to serve Raon-1 nano through an OpenAI-compatible API. The example below targets one NVIDIA B300 GPU with BF16 weights.

Install from the raon-1-nano branch of KRAFTON's vLLM:

uv venv --python 3.12
source .venv/bin/activate

VLLM_USE_PRECOMPILED=1 uv pip install \
    "vllm @ git+https://github.com/krafton-ai/vllm.git@raon-1-nano"

Start the server:

vllm serve KRAFTON/Raon-1-nano \
    --host 127.0.0.1 \
    --port 8000 \
    --served-model-name Raon-1-nano \
    --trust-remote-code \
    --dtype bfloat16 \
    --tensor-parallel-size 1 \
    --pipeline-parallel-size 1 \
    --max-model-len 262144 \
    --gpu-memory-utilization 0.85 \
    --reasoning-parser raon_1 \
    --enable-auto-tool-choice \
    --tool-call-parser raon_1 \
    --default-chat-template-kwargs '{
      "enable_thinking": true,
      "truncate_history_thinking": false
    }'

Send a request from another terminal:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "Raon-1-nano",
      "messages": [
        {"role": "user", "content": "Tell me about KRAFTON AI."}
      ],
      "max_tokens": 4096,
      "temperature": 0.1,
      "presence_penalty": 1.5,
      "chat_template_kwargs": {
        "enable_thinking": true,
        "truncate_history_thinking": false
      }
    }'

max_tokens includes both thinking and the final answer.

llama.cpp

Run the GGUF model on Apple silicon (Metal), NVIDIA GPUs (CUDA), AMD GPUs (Vulkan), or CPU.

Model: KRAFTON/Raon-1-nano-gguf

Build the Raon-1 nano branch of KRAFTON's llama.cpp: On Windows, see the build guide.

git clone -b raon-1-nano-preview \
    https://github.com/krafton-ai/llama.cpp
cd llama.cpp

# Apple silicon: no flag needed (Metal is on by default).
# NVIDIA: add -DGGML_CUDA=ON. AMD: add -DGGML_VULKAN=ON.
cmake -B build
cmake --build build --config Release -j --target llama-cli llama-server

On Windows, run these in the Developer PowerShell for Visual Studio; the binaries are in build\bin\Release\ (for example .\build\bin\Release\llama-cli.exe).

Start a terminal chat. The model downloads automatically.

./build/bin/llama-cli \
    -hf KRAFTON/Raon-1-nano-gguf:Q4_K_M \
    --ctx-size 65536 \
    --reasoning on \
    --chat-template-kwargs '{"enable_thinking": true}'

Or start an OpenAI-compatible server:

./build/bin/llama-server \
    -hf KRAFTON/Raon-1-nano-gguf:Q4_K_M \
    --host 127.0.0.1 \
    --port 8000 \
    --alias Raon-1-nano \
    --ctx-size 65536 \
    --parallel 1 \
    --reasoning on \
    --chat-template-kwargs '{"enable_thinking": true}'

Send a request from another terminal:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "Raon-1-nano",
      "messages": [
        {"role": "user", "content": "Tell me about KRAFTON AI."}
      ],
      "temperature": 0.1,
      "presence_penalty": 1.5
    }'

The thinking text is returned in reasoning_content, the answer in content.

MLX

Run the 4-bit MLX model (GPTQ) on Apple silicon.

Model: KRAFTON/Raon-1-nano-mlx

The model code ships in the MLX repository (raon_mlx_model.py), so pass trust_remote_code=True. It includes our fused Metal kernels for decode and prefill; they are enabled only with mlx==0.32.4 (other versions run a slower path).

Install the dependencies with Python 3.11+:

python3 -m venv .venv
source .venv/bin/activate

python -m pip install \
    "mlx==0.32.4" \
    "mlx-lm==0.32.2" \
    "transformers==5.20.2"

Run in Python:

from mlx_lm import load, generate

model, tokenizer = load(
    "KRAFTON/Raon-1-nano-mlx",
    trust_remote_code=True,
)

messages = [
    {"role": "user", "content": "Tell me about KRAFTON AI."}
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=4096,
)
print(response)

Or start an OpenAI-compatible server:

python -m mlx_lm server \
    --model KRAFTON/Raon-1-nano-mlx \
    --trust-remote-code \
    --host 127.0.0.1 \
    --port 8000 \
    --chat-template-args '{"enable_thinking": true}'

Send a request from another terminal:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "KRAFTON/Raon-1-nano-mlx",
      "messages": [
        {"role": "user", "content": "Tell me about KRAFTON AI."}
      ]
    }'

The thinking text is returned in reasoning, the answer in content.

BF16 (no quantization)

The model above is 4-bit. To run without quantization, export a BF16 MLX model from KRAFTON/Raon-1-nano with the model code in the MLX repository (about 6.3 GB; decode is slower than 4-bit):

import json, shutil, sys
from pathlib import Path

import mlx.core as mx
from mlx.utils import tree_flatten
from huggingface_hub import snapshot_download

code = Path(snapshot_download("KRAFTON/Raon-1-nano-mlx", ignore_patterns=["model.safetensors"]))
original = Path(snapshot_download("KRAFTON/Raon-1-nano"))
out = Path("Raon-1-nano-mlx-bf16")

sys.path.insert(0, str(code))
import raon_mlx_model

model = raon_mlx_model.load_model(original, dtype=mx.bfloat16)
out.mkdir()
mx.save_safetensors(str(out / "model.safetensors"), dict(tree_flatten(model.parameters())))
for f in code.iterdir():
    if f.is_file() and f.name not in ("model.safetensors", "config.json", "README.md"):
        shutil.copy(f, out / f.name)
config = json.loads((code / "config.json").read_text())
config.pop("quantization")
(out / "config.json").write_text(json.dumps(config, indent=2))

Then use the commands above with the local folder in place of the repository id, for example load("Raon-1-nano-mlx-bf16", trust_remote_code=True) or python -m mlx_lm server --model Raon-1-nano-mlx-bf16 --trust-remote-code ....

LiteRT-LM

Run the LiteRT-LM bundle (4-bit weights, fp16 activations) with the official litert-lm package; no runtime changes are needed. Checked on Apple silicon (GPU).

Model: KRAFTON/Raon-1-nano-litert-lm

This bundle has a 4,096-token context and answers without thinking.

Install the dependencies with Python 3.12:

python3.12 -m venv .venv
source .venv/bin/activate

python -m pip install "litert-lm==0.18.1"

Import the model once from the LiteRT-LM repository, then chat:

litert-lm import \
    --from-huggingface-repo KRAFTON/Raon-1-nano-litert-lm \
    model.litertlm \
    raon-1-nano

litert-lm run raon-1-nano \
    --backend gpu \
    --max-num-tokens 4096 \
    --prompt "Tell me about KRAFTON AI."

The first run compiles the GPU program (about 30-60 seconds) and caches it next to the model; later runs start quickly.

Or start an OpenAI-compatible server. Save this as raon-litert.json:

{
  "models": {
    "raon-1-nano": {
      "backend": "gpu",
      "max_num_tokens": 4096
    }
  }
}

litert-lm serve \
    --config raon-litert.json \
    --host 127.0.0.1 \
    --port 8000

Send a request from another terminal:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "raon-1-nano",
      "messages": [
        {"role": "user", "content": "Tell me about KRAFTON AI."}
      ]
    }'

License

Hugging Face model card / README notice License - Apache 2.0

The Raon model, including its model weights, configuration files, and other model components made available for download through this repository, is licensed under the Apache License, Version 2.0. Subject to that license, you may download, use, modify, and redistribute the model and its weights, including for commercial purposes.

Krafton and its licensors retain ownership of their respective intellectual property. The Apache License grants the rights stated in that license and does not transfer ownership. Rights in third-party materials remain subject to their applicable terms.

The Apache License does not extend to Krafton’s proprietary training datasets, training methodologies, undistributed software, or other proprietary materials not included in this repository.