Instructions to use Yong-Hoon/MIAI_VLM_0.2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Yong-Hoon/MIAI_VLM_0.2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Yong-Hoon/MIAI_VLM_0.2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Yong-Hoon/MIAI_VLM_0.2")
model = AutoModelForMultimodalLM.from_pretrained("Yong-Hoon/MIAI_VLM_0.2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Yong-Hoon/MIAI_VLM_0.2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Yong-Hoon/MIAI_VLM_0.2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yong-Hoon/MIAI_VLM_0.2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Yong-Hoon/MIAI_VLM_0.2

SGLang

How to use Yong-Hoon/MIAI_VLM_0.2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Yong-Hoon/MIAI_VLM_0.2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yong-Hoon/MIAI_VLM_0.2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Yong-Hoon/MIAI_VLM_0.2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Yong-Hoon/MIAI_VLM_0.2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Yong-Hoon/MIAI_VLM_0.2 with Docker Model Runner:
```
docker model run hf.co/Yong-Hoon/MIAI_VLM_0.2
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

MIAI-VLM 0.2

Korean/English vision-language model: google/gemma-4-E4B-it fine-tuned with LoRA on 54.0M image-text and text samples (191 datasets, 46% Korean). Snapshot of an ongoing run at step 1,000,000 of 2,531,640 (epoch 1.19 of 3). Merged bf16 weights are at the root; the LoRA adapter is in adapter/.


Base	google/gemma-4-E4B-it (8.0B params incl. vision/audio towers)
Fine-tuning	LoRA r=32, α=64 on all linear layers of the language model (69.8M trainable params); vision/audio towers frozen
Data	54.0M samples · image-text 29% / text 71% · Korean 46% / English 54%
Compute	16 × RTX 3090 (2 nodes), effective batch 64, lr 2e-4 cosine, 1,024-token sequences, bf16 · ≈ 61 days for this snapshot
Framework	LLaMA-Factory 0.9.6 · transformers 5.6 · PEFT 0.18

Training loss 0.921 (EMA) at this snapshot; per-100-step logs in training/trainer_state.json, config in training/train_config.yaml.

Usage

The model was trained with the system prompt You are a helpful assistant. and enable_thinking=True; use the same format. It answers directly after an empty thought channel, which is stripped below.

import re, torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "Yong-Hoon/MIAI_VLM_0.2"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto").eval()

def chat(question, image=None, max_new_tokens=256):
    content = ([{"type": "image", "image": image}] if image is not None else []) + [{"type": "text", "text": question}]
    messages = [
        {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
        {"role": "user", "content": content},
    ]
    inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
                                           return_dict=True, return_tensors="pt", enable_thinking=True).to(model.device)
    out = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False)
    text = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
    return re.sub(r"<\|channel\>thought\n.*?<channel\|>", "", text, flags=re.S).replace("<turn|>", "").strip()

print(chat("이 사진에 무엇이 보이나요? 두 문장으로 설명해주세요.", image=Image.open("photo.jpg")))

Adapter only: PeftModel.from_pretrained(base_model, "Yong-Hoon/MIAI_VLM_0.2", subfolder="adapter").

Files

model-*.safetensors (merged, ~16 GB) · tokenizer/processor configs · adapter/ (LoRA, ~280 MB) · training/ (config + loss log) · assets/ (charts)

Intermediate snapshot; later snapshots will follow as new versions. Governed by the Gemma license. Built with LLaMA-Factory. Developed by Yong-Hoon (KETI).

Downloads last month: -

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for Yong-Hoon/MIAI_VLM_0.2

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Adapter

(337)

this model