Instructions to use K-Ai-Official/K-Oss-8-Mini-Instruct-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use K-Ai-Official/K-Oss-8-Mini-Instruct-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="K-Ai-Official/K-Oss-8-Mini-Instruct-8B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("K-Ai-Official/K-Oss-8-Mini-Instruct-8B")
model = AutoModelForMultimodalLM.from_pretrained("K-Ai-Official/K-Oss-8-Mini-Instruct-8B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use K-Ai-Official/K-Oss-8-Mini-Instruct-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "K-Ai-Official/K-Oss-8-Mini-Instruct-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "K-Ai-Official/K-Oss-8-Mini-Instruct-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/K-Ai-Official/K-Oss-8-Mini-Instruct-8B

SGLang

How to use K-Ai-Official/K-Oss-8-Mini-Instruct-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "K-Ai-Official/K-Oss-8-Mini-Instruct-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "K-Ai-Official/K-Oss-8-Mini-Instruct-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "K-Ai-Official/K-Oss-8-Mini-Instruct-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "K-Ai-Official/K-Oss-8-Mini-Instruct-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use K-Ai-Official/K-Oss-8-Mini-Instruct-8B with Docker Model Runner:
```
docker model run hf.co/K-Ai-Official/K-Oss-8-Mini-Instruct-8B
```

K-Oss 8 Mini Instruct

K-Oss 8 Mini Instruct is 8B Parameter Model, instruction-tuned member of the K-Oss 8 family by K-Ai (Kortex AI). It accepts text and images and answers in English, Indonesian (Bahasa Indonesia) and Japanese (日本語).

It is fine-tuned from Qwen/Qwen3-VL-8B-Instruct with QLoRA (language-model layers trained, vision tower frozen). This repository contains the merged 16-bit weights.

About the K-Oss 8 family. K-Oss 8 (120B total / 12B active parameters, based on NVIDIA Nemotron 3.5 Super VL) is a separate, larger model. K-Oss 8 Mini Instruct is built on Qwen3-VL-8B-Instruct, uses no Nemotron weights and is not distilled from K-Oss 8. It shares the family name, goals and language focus; do not assume it matches the 120B model.

Language support

English, Indonesian and Japanese are the target languages. Each was trained on and tested separately (passed/total on small held-out sets):

Language	In-domain tests (fine-tuned)	In-domain tests (base)	General Q&A (fine-tuned)	Replies in the right language
Indonesian	17/20	15/20	16/20	19/20
English	17/20	17/20	17/20	19/20
Japanese	14/20	12/20	13/20	16/20

Model details


Developer	K-Ai (Kortex AI)
Base model	Qwen/Qwen3-VL-8B-Instruct (about 8.8B parameters)
Fine-tuning	QLoRA, r=16, alpha=16
Input / output	Text and images in, text out
Languages	English, Indonesian, Japanese
License	Apache-2.0 (follows the base model)

Training procedure

Method: supervised fine-tuning (TRL SFTTrainer, Unsloth vision collator) on 4-bit quantized base weights, then merged to 16-bit
Trainable: attention and MLP modules of the language model; frozen: vision tower
Hyperparameters: [[steps]] steps, batch size 1 x gradient accumulation 8 (effective 8), learning rate [[1e-4]], AdamW 8-bit, weight decay 0.01, fp16, max sequence length 1536, images resized to at most 640 px, seed 3407
Hardware: Google Colab, one NVIDIA T4 (16 GB); training time [[minutes]]; loss [[start]] -> [[end]]

Training data

[[total]] examples, mostly synthetic (dataset kai-synth-v1, generated by a script with exactly computed answers), in all three languages:

Images: OCR (codes, numbers, words), receipts (Rp / $ / ¥), bar charts, tables, forms, math in images, shape counting
Text: word problems, general facts, translation between the three languages, identity, and a small set of refusal / benign security examples
Held out for testing: the last question template per task and language, 20% of OCR words, selected facts and phrases
Some text examples (identity, translation phrases, facts, refusal and benign answers) were written with AI assistance and were not reviewed by native speakers.
No public instruction datasets were mixed in.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained(
    "Keanualfahd/K-Oss-8-Mini-Instruct", torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("Keanualfahd/K-Oss-8-Mini-Instruct")

messages = [{"role": "user", "content": [
    {"type": "image", "image": "https://example.com/receipt.jpg"},
    {"type": "text", "text": "Berapa total pada struk ini?"},
]}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
).to(model.device)
out = model.generate(**inputs, max_new_tokens=128)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Intended use

K-Oss 8 Mini Instruct is a multilingual instruction-following model developed by K-Ai, based on Qwen3-VL-8B.

Primary Use Cases

General-purpose conversational AI.
Indonesian, English, and Japanese language assistance.
Question answering and information assistance.
Text generation, summarization, and basic reasoning.
Educational assistance and learning support.
Multimodal interaction, subject to the capabilities retained from the base architecture.

Target Users

Developers integrating AI into applications.
Students and educators.
AI researchers and enthusiasts.
Users exploring multilingual AI systems.

Out-of-Scope Uses

Medical, legal, or financial decisions without qualified human oversight.
Generating harmful, deceptive, or misleading content.
High-stakes automated decision-making.

This model is intended for research, experimentation, and general-purpose AI applications. Its outputs should be reviewed for accuracy and reliability.

License and attribution

Apache-2.0, following Qwen3-VL-8B-Instruct by the Qwen team (Alibaba Cloud). K-Oss 8's own license terms follow its Nemotron base and are separate. Fine-tuned with Unsloth and TRL.

Ringkasan (Bahasa Indonesia)

K-Oss 8 Mini Instruct adalah model multimodal kecil dari keluarga K-Oss 8 buatan K-Ai. Model ini membaca teks dan gambar dan menjawab dalam bahasa Inggris, Indonesia, dan Jepang. Hasil fine-tuning dari Qwen3-VL-8B-Instruct, sebagian besar dilatih dengan data sintetis. Status: Public Beta, belum diuji pada gambar nyata dalam skala besar.

概要（日本語）

K-Oss 8 Mini Instructは、K-AiのK-Oss 8ファミリーの小型マルチモーダルモデルです。テキストと画像を理解し、英語・インドネシア語・日本語で回答します。Qwen3-VL-8B-Instructをファインチューニングしたもので、学習データの大部分は合成データです。実際の画像での大規模な検証はまだ行っていません。

Downloads last month: -

Model tree for K-Ai-Official/K-Oss-8-Mini-Instruct-8B

Base model

Qwen/Qwen3-VL-8B-Instruct

Finetuned

(710)

this model

K-Ai-Official
/

K-Oss-8-Mini-Instruct-8B