Instructions to use Qwen/Qwen3.8-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen3.8-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Qwen/Qwen3.8-27B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Qwen/Qwen3.8-27B")
model = AutoModelForMultimodalLM.from_pretrained("Qwen/Qwen3.8-27B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Qwen/Qwen3.8-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3.8-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen3.8-27B

SGLang

How to use Qwen/Qwen3.8-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3.8-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen3.8-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen3.8-27B with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen3.8-27B
```

Very good 🥰 but thinks too much 🧠

#82

by auf1r2 - opened 3 days ago

Discussion

auf1r2

3 days ago

•

edited 3 days ago

Hey Qwenny team!

Thank you for sharing this amazing model to comunity! 🥰

I like this model's output but it thinks too much on my taste.

Example
Setup: llama.cpp + Q8_K_XL quant from Unsloth + 4 tools (web search, web fetch, hugging face, date_time)
Prompt: Does Qwen3.8 GGUF from Unsloth support MTP?

Results
All correct, it found 2 Qwen3.8 models from Unsloth on the Huggingface and answered correctly, but...
It used half hundred tool calls + about 85000 tokens out of 256000 context! While it was enough to stop on first 5 tool calls it made to reach this conclusion.

loumadev

3 days ago

You have to lower the reasoning effort, which is xhigh by default. Configure via --chat-template-kwargs '{"reasoning_effort":"xhigh"}', where reasoning_effort can be "xhigh", "medium" or "low".

From official model card:

Qwen3.8 comes with official support for reasoning_effort, which can be used to adjust reasoning depth and control cost:

xhigh (default): for complex tasks demanding thorough analysis

medium: balancing accuracy and speed

low: efficient reasoning optimizing for speed and cost

LuffyTheFox

3 days ago

Hey Qwenny team!

Thank you for sharing this amazing model to comunity! 🥰

I like this model's output but it thinks too much on my taste.

Example
Setup: llama.cpp + Q8_K_XL quant from Unsloth + 4 tools (web search, web fetch, hugging face, date_time)
Prompt: Does Qwen3.8 GGUF from Unsloth support MTP?

Results
All correct, it found 2 Qwen3.8 models from Unsloth on the Huggingface and answered correctly, but...
It used half hundred tool calls + about 85000 tokens out of 256000 context! While it was enough to stop on first 5 tool calls it made to reach this conclusion.

Here the mathematical reason why it happens: https://huggingface.co/Qwen/Qwen3.8-27B/discussions/76

MarbuelB

3 days ago

•

edited 3 days ago

I agree ... waaaay too much thinking, even at "low" settings. My local vLLM server (serving FP8 model) is timing out on simple tasks that were perfectly doable with Qwen 3.6. Unusable, unless you block thinking completely. Could there be a difference in how it is served? When testing with Ollama (also FP8 model), it felt like it does less thinking than with vLLM (I did not not do any strict comparison or systematic tests).

YUYUE08

3 days ago

Same,I am using vllm and it thinks tooooo much for low effort.It took me almost an hour to wrote a Mario game.And most thinking is useless.Great model though.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment