Instructions to use peterbuitho/VietPoet-Qwen3.5-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use peterbuitho/VietPoet-Qwen3.5-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="peterbuitho/VietPoet-Qwen3.5-9B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("peterbuitho/VietPoet-Qwen3.5-9B")
model = AutoModelForMultimodalLM.from_pretrained("peterbuitho/VietPoet-Qwen3.5-9B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use peterbuitho/VietPoet-Qwen3.5-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "peterbuitho/VietPoet-Qwen3.5-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "peterbuitho/VietPoet-Qwen3.5-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/peterbuitho/VietPoet-Qwen3.5-9B

SGLang

How to use peterbuitho/VietPoet-Qwen3.5-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "peterbuitho/VietPoet-Qwen3.5-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "peterbuitho/VietPoet-Qwen3.5-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "peterbuitho/VietPoet-Qwen3.5-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "peterbuitho/VietPoet-Qwen3.5-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use peterbuitho/VietPoet-Qwen3.5-9B with Docker Model Runner:
```
docker model run hf.co/peterbuitho/VietPoet-Qwen3.5-9B
```

VietPoet Qwen3.5-9B

A Qwen3.5-9B fine-tuned (QLoRA, 8,000 poems, 2 epochs, LoRA merged into the weights) to write Vietnamese lục bát poems. This is the 16-bit (bf16) safetensors model, 18 GB. For LM Studio / llama.cpp use the GGUF version; for vLLM or transformers use this one.

It is meant to be used with the line-by-line sampler and rule checker from github.com/peterbuitho/ThoLucBat. Used on its own it writes the right shape but breaks the tone rules more often.

What it does and does not do

On 100 held-out "8 câu" prompts (rule score 0 to 1: 0.1 length + 0.3 tone + 0.6 rhyme):

	raw model	with line-by-line sampler (16 samples per line)
mean rule score	0.829	0.994
fully valid poems	14%	98%

Raw, 10% of the bát lines break the tone rule for the 6th/8th syllable. The sampler generates each line several times and keeps the best one that satisfies the rules.

These numbers measure form, not poetry. Poems are correct lục bát but the meaning is often loose or off-topic (training prompts only had the poem title as topic). Judge the poetry yourself.

Prompt format

Qwen chat format with thinking off. The sampler appends the poem line by line to the assistant turn:

<|im_start|>system
Bạn là nhà thơ Việt Nam chuyên sáng tác thơ lục bát.<|im_end|>
<|im_start|>user
Viết một bài thơ lục bát 8 câu về mùa thu quê em.<|im_end|>
<|im_start|>assistant
<think>

</think>

The request wording varies (several templates); they are in app/prompts.py in the GitHub repo.

Training

QLoRA (4-bit base), r=16, alpha=16, lr 2e-4, 2 epochs, batch 2 x grad-accum 4, max length 1024, Unsloth 2026.9.7. The full parameters are in the repo (runs/sft-9b-v1/train_config.json).

Credits

Training data: phamson02/vietnamese-poetry-corpus (CC BY 4.0), filtered to poems that pass a lục bát rule checker.
Base model: Qwen/Qwen3.5-9B (Apache-2.0).
Scoring idea: Vietnamese Poem Generation & the Prospect of Cross-Language Poem-to-Poem Translation (arXiv:2401.01078).

Downloads last month: 395

Safetensors

Model size

10B params

Tensor type

BF16

F32

Model tree for peterbuitho/VietPoet-Qwen3.5-9B

Base model

Qwen/Qwen3.5-9B-Base

Finetuned

Qwen/Qwen3.5-9B

Finetuned

(862)

this model

Dataset used to train peterbuitho/VietPoet-Qwen3.5-9B

Paper for peterbuitho/VietPoet-Qwen3.5-9B

Vietnamese Poem Generation & The Prospect Of Cross-Language Poem-To-Poem Translation

Paper • 2401.01078 • Published Jan 4, 2024