Instructions to use ManniX-ITA/Qwen3.6-27B-A3B-Coder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ManniX-ITA/Qwen3.6-27B-A3B-Coder with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ManniX-ITA/Qwen3.6-27B-A3B-Coder")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ManniX-ITA/Qwen3.6-27B-A3B-Coder")
model = AutoModelForMultimodalLM.from_pretrained("ManniX-ITA/Qwen3.6-27B-A3B-Coder", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ManniX-ITA/Qwen3.6-27B-A3B-Coder with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ManniX-ITA/Qwen3.6-27B-A3B-Coder"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/Qwen3.6-27B-A3B-Coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ManniX-ITA/Qwen3.6-27B-A3B-Coder

SGLang

How to use ManniX-ITA/Qwen3.6-27B-A3B-Coder with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ManniX-ITA/Qwen3.6-27B-A3B-Coder" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/Qwen3.6-27B-A3B-Coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ManniX-ITA/Qwen3.6-27B-A3B-Coder" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/Qwen3.6-27B-A3B-Coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ManniX-ITA/Qwen3.6-27B-A3B-Coder with Docker Model Runner:
```
docker model run hf.co/ManniX-ITA/Qwen3.6-27B-A3B-Coder
```

Best agentic performing variant of qwen3.6 A3b family

by repla73 - opened 15 days ago

Discussion

repla73

15 days ago

I've created a real world usage agentic benchmark environment to test similar sized models.
Pi harness inside Ubuntu24.04 Docker container.
I don't know how you did it but this slimmed version performed even better than the same models 35b variant and also all others.
The best part is the model runs on pure ARM Ampere A1 4oCpu's and no gpu with PP=35 and PG=15tok/sec output!
Thanks for the great work and I'm waiting for Qwen3.8 version of this model!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment