Instructions to use Qwen/Qwen3-Coder-480B-A35B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Qwen/Qwen3-Coder-480B-A35B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Qwen/Qwen3-Coder-480B-A35B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-Coder-480B-A35B-Instruct") model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-Coder-480B-A35B-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Qwen/Qwen3-Coder-480B-A35B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Qwen/Qwen3-Coder-480B-A35B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen3-Coder-480B-A35B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Qwen/Qwen3-Coder-480B-A35B-Instruct
- SGLang
How to use Qwen/Qwen3-Coder-480B-A35B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Qwen/Qwen3-Coder-480B-A35B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen3-Coder-480B-A35B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Qwen/Qwen3-Coder-480B-A35B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Qwen/Qwen3-Coder-480B-A35B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Qwen/Qwen3-Coder-480B-A35B-Instruct with Docker Model Runner:
docker model run hf.co/Qwen/Qwen3-Coder-480B-A35B-Instruct
Bug Report: qwen3-coder Outputs Tool Call Argument in Incorrect Type for String-Encoded JSON
I encountered an issue where qwen3-coder outputs a tool call argument with an incorrect type.
Specifically, for a tool schema that expects a string value (which contains a JSON string), the model incorrectly outputs it as a JSON object instead of a string.
this is the tool
def write(file:str, content: str)
request example (for openrouter)
{
"model": "qwen/qwen3-coder",
"messages": [
{
"role": "user",
"content": "create a pure json content without codeblocks, with field a , a is a string of escaped json object that contains key b , b is string , content is `cccc` and write file 'a.json'"
}
],
"tools": [
{
"name": "Write",
"description": "Writes a file to the local filesystem.\n\nUsage:\n- This tool will overwrite the existing file if there is one at the provided path.\n- If this is an existing file, you MUST use the Read tool first to read the file's contents. This tool will fail if you did not read the file first.\n- ALWAYS prefer editing existing files in the codebase. NEVER write new files unless explicitly required.\n- NEVER proactively create documentation files (*.md) or README files. Only create documentation files if explicitly requested by the User.\n- Only use emojis if the user explicitly requests it. Avoid writing emojis to files unless asked.",
"input_schema": {
"type": "object",
"properties": {
"file_path": {
"type": "string",
"description": "The absolute path to the file to write (must be absolute, not relative)"
},
"content": {
"type": "char",
"description": "The content to write to the file, should be string"
}
},
"required": [
"file_path",
"content"
],
"additionalProperties": false,
}
}
],
"max_tokens": 4200
}
Expected response
{'choices': [{'message': {'role': 'assistant', 'content': '', 'tool_calls': [{'name': 'Write', 'arguments': '{"content": "{\\"pattern\\": \\"package\\", \\"path\\": \\"/xxx/ClientExample.java\\"}", "file_path": "a.json"}'}]}}]}
{"value": "{\n \"a\": \"b\"\n}"}
actual output
{'choices': [{'message': {'role': 'assistant', 'content': '', 'tool_calls': [{'name': 'Write', 'arguments': '{"content": {"pattern": "package", "path": "/Users/qixiang/Projects/solutions/testmcp/src/main/java/com/tpp/solution/CallTpp/TppClientExample.java"}, "file_path": "a.json"}'}]}}]}
Note: The content field is parsed into an object, which violates the tool's schema that defines content as a string.
๐ Root Cause (Suspected)
In the parser code (screenshot below), it seems that string fields are not validated for type correctness โ if the content looks like a JSON object, param value is deserialized, and the parsing code will return directly, will value type converting
For fields of type object, this behavior is correct.
But for fields explicitly defined as string, the parser ignores value type validating โ it should make sure the raw string form.

