Instructions to use Qwen/Qwen3-30B-A3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen3-30B-A3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Qwen/Qwen3-30B-A3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-30B-A3B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-30B-A3B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
AMD Developer Cloud
Local Apps Settings

vLLM

How to use Qwen/Qwen3-30B-A3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3-30B-A3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3-30B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen3-30B-A3B

SGLang

How to use Qwen/Qwen3-30B-A3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3-30B-A3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3-30B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen3-30B-A3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3-30B-A3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen3-30B-A3B with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen3-30B-A3B
```

image support

by kuliev-vitaly - opened Apr 29, 2025

Discussion

kuliev-vitaly

Apr 29, 2025

•

edited Apr 29, 2025

According to blog on github qwen 3 support text, image, video and audio as input. According to model card it support only text as input. Does it support image as input? How to start model with image adapter?

kuliev-vitaly

Apr 30, 2025

https://chat.qwen.ai/
This model works with images in qwen chat.

pySilver

Apr 30, 2025

I came to ask the same question. Looks like released model is only text generation, while online supports multimodality.

urroxyz

Apr 30, 2025

Is this model—the open-weights one—trained for handling those inputs? If so, could we use an adapter or additional encoder with it?

gelim

May 13, 2025

yes, wondering if there will be multimodal projector files to be shared later?
See as an example:

akierum

May 23, 2025

So no way to upload images to this model locally? What kind of nonsense is this? Open sourced ? Even qwq does not support images?

viggypiggy

Jun 1, 2025

I guess qwen chat has proprietary image encoder? It will be great if they share this part.
Qwen3 models in qwen chat solves my usecase very well (involves image).

Hope they share this soon.

akierum

Jun 1, 2025

Better use devstral from unsloth it has vision, but its for coding. First vision + coding model, set temp to 0.05

shumybest

Jun 25, 2025

but there's no clue why https://chat.qwen.ai/ 's qwen3 works with image, I've tested on aliyun's qwen3, no image support, anyone knows why chat.qwen.ai can support image on qwen3?

shamil2398

Sep 7, 2025

but there's no clue why https://chat.qwen.ai/ 's qwen3 works with image, I've tested on aliyun's qwen3, no image support, anyone knows why chat.qwen.ai can support image on qwen3?

Same question, I suppose chat.qwen uses a tool call for images, it also generates images while you on qwen3. So, probably it is just tool call of QwenVL-Max

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment