Qwen3.5-0.8B-q4f16_1-MLC

This repository is a mirror of mlc-ai/Qwen3.5-0.8B-q4f16_1-MLC in MLC format q4f16_1. Compatible with MLC-LLM and WebLLM.

Upstream Details

• Original Base Model: Qwen/Qwen3.5-0.8B
• MLC Source Repository: mlc-ai/Qwen3.5-0.8B-q4f16_1-MLC
• License: Apache-2.0

Example Usage

Chat via MLC-LLM

mlc_llm chat HF://hb-dev/Qwen3.5-0.8B-q4f16_1-MLC

REST Server

mlc_llm serve HF://hb-dev/Qwen3.5-0.8B-q4f16_1-MLC

Python API

from mlc_llm import MLCEngine

model = "HF://hb-dev/Qwen3.5-0.8B-q4f16_1-MLC"
engine = MLCEngine(model)

for response in engine.chat.completions.create(
    messages=[{"role": "user", "content": "What is the meaning of life?"}],
    model=model,
    stream=True,
):
    for choice in response.choices:
        print(choice.delta.content, end="", flush=True)
print("\n")

engine.terminate()