Instructions to use upwitu/qwen3-4b-sft-all with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use upwitu/qwen3-4b-sft-all with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="upwitu/qwen3-4b-sft-all")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("upwitu/qwen3-4b-sft-all")
model = AutoModelForCausalLM.from_pretrained("upwitu/qwen3-4b-sft-all", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use upwitu/qwen3-4b-sft-all with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "upwitu/qwen3-4b-sft-all"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "upwitu/qwen3-4b-sft-all",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/upwitu/qwen3-4b-sft-all

SGLang

How to use upwitu/qwen3-4b-sft-all with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "upwitu/qwen3-4b-sft-all" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "upwitu/qwen3-4b-sft-all",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "upwitu/qwen3-4b-sft-all" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "upwitu/qwen3-4b-sft-all",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use upwitu/qwen3-4b-sft-all with Docker Model Runner:
```
docker model run hf.co/upwitu/qwen3-4b-sft-all
```

Qwen3-4B CAR-Bench Winner-Inspired SFT (Merged 16-Bit Weights)

English | Tiếng Việt

English

This model contains the full 16-bit (bfloat16) merged weights of Qwen3-4B-Instruct fine-tuned on the multi-task CAR-Bench Winner Dataset. The training trajectories synthesize safety invariants and execution patterns from the top three winning architectures of the IJCAI 2026 CAR-Bench competition:

10CARS (Innovation Award): Read-Before-Write (AUT-POL:001), Confirmation Gate (AUT-POL:002), Actuator Boundary Gates (AUT-POL:007/008).
FreudeDrive (Track 2 Champion): Multi-Role Concurrency, 4-step Chain of Thought ([Context Audit], [Policy Check], [Tool Selection & Provenance], [Execution Plan]), and 2-step ID Provenance (AUT-POL:018).
Proxima Ultra (Track 2 Champion): Programmatic CodeAct, Policy as Code, Coroutine Bridge, and Unknown-Value Sentinels.

Data Lineage & Training Architecture

Click to view Mermaid diagram source code

flowchart TD
    subgraph Papers["1. Architectural Foundations (IJCAI 2026 Winners)"]
        P1["10CARS<br/>(L1-L3 Pre-Flight Gate & Read-Before-Write)"]
        P2["FreudeDrive<br/>(4-Step CoT & Parameter Provenance)"]
        P3["Proxima Ultra<br/>(Programmatic CodeAct & Policy-as-Code)"]
    end

    subgraph DataGen["2. Synthesis & Sanitization Engine"]
        GEN["sft_generator (Async Multi-Worker Engine)"]
        SAN["scripts/sanitize_dataset.py<br/>(Tool ID Binding & Schema Repair)"]
        D1["carbench_sft_multirole_json.jsonl (1,324 samples)"]
        D2["carbench_sft_codeact_python.jsonl (1,443 samples)"]
        GEN --> SAN
        SAN --> D1
        SAN --> D2
    end

    subgraph Training["3. Supervised Fine-Tuning (SFT)"]
        BASE["Base Model: Qwen3-4B-Instruct-2507"]
        LORA["LoRA Fine-Tuning (r=16, alpha=32)<br/>Targets: q, k, v, o, gate, up, down_proj<br/>Loss: 0.7222 | Epochs: 2.0 | 148 steps"]
        MERGE["Merged 16-Bit Model (BF16 Safetensors, 7.6GB)"]
        BASE --> LORA --> MERGE
    end

    Papers --> DataGen
    D1 --> LORA
    D2 --> LORA

In-Cabin Inference & vLLM Serving Pipeline

Click to view Mermaid diagram source code

sequenceDiagram
    autonumber
    actor Driver as Driver / Evaluator
    participant vLLM as vLLM Server (Port 8000)
    participant Model as Qwen3-4B Merged SFT Model
    participant Parser as Hermes Tool Call Parser
    participant Vehicle as In-Cabin Vehicle System (58 Tools)

    Driver->>vLLM: POST /v1/chat/completions (Voice Request + Tools)
    vLLM->>Model: Forward Context + Chat Template
    Note over Model: Generates 4-Step CoT reasoning:<br/>[Context Audit] -> [Policy Check] -> [Tool Selection] -> [Plan]
    Model-->>Parser: Tool call tokens
    Parser-->>vLLM: Structured tool_calls payload
    vLLM-->>Driver: Emit Function Call: get_climate_settings()
    Driver->>Vehicle: Execute get_climate_settings()
    Vehicle-->>Driver: {"status": "success", "temperature": 19.0}
    Driver->>vLLM: POST /v1/chat/completions (Tool Result)
    vLLM->>Model: Forward Tool Result
    Model-->>vLLM: Final voice response: "Cabin temperature adjusted to 22°C."
    vLLM-->>Driver: Natural Speech Output

Quickstart: Serving with vLLM

Run the merged model locally with the official vLLM OpenAI-compatible server:

vllm serve upwitu/qwen3-4b-sft-all \
    --port 8000 \
    --dtype bfloat16 \
    --max-model-len 8192 \
    --gpu-memory-utilization 0.90 \
    --tool-call-parser hermes

Python Inference with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "upwitu/qwen3-4b-sft-all"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a professional in-cabin autonomous vehicle AI assistant. Follow 10CARS safety rules: inspect before mutating, ask confirmation for safety-critical operations, and respect operational limits."},
    {"role": "user", "content": "Set cabin temperature to 23°C."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Tiếng Việt

Mô hình chứa toàn bộ trọng số 16-bit hợp nhất (bfloat16) của Qwen3-4B-Instruct được tinh chỉnh SFT trên tập dữ liệu CAR-Bench Winner Dataset. Dữ liệu huấn luyện chắt lọc các quy tắc an toàn và mô thức điều phối từ ba đội tuyển vô địch cuộc thi CAR-Bench (IJCAI 2026):

10CARS (Giải Sáng Tạo): Đọc trước khi ghi (AUT-POL:001), cổng xác nhận (AUT-POL:002), giới hạn tham số vận hành (AUT-POL:007/008).
FreudeDrive (Đồng Quán Quân Track 2): Phân vai đồng thời, chuỗi suy luận CoT 4 bước (Context Audit $\to$ Policy Check $\to$ Tool Selection & Provenance $\to$ Execution Plan), xác thực nguồn gốc ID 2 bước (AUT-POL:018).
Proxima Ultra (Đồng Quán Quân Track 2): Lập trình thực thi CodeAct, chính sách trong mã nguồn, cầu nối Coroutine, và bộ bắt Sentinel chống ảo giác.

Nguồn Gốc Dữ Liệu & Kiến Trúc Huấn Luyện

Bấm để xem mã nguồn biểu đồ Mermaid

flowchart TD
    subgraph NenTang["1. Nền Tảng Lý Thuyết (3 Đội Vô Địch IJCAI 2026)"]
        P1["10CARS<br/>(Cổng Tiền Bay L3 & Đọc Trước Khi Ghi)"]
        P2["FreudeDrive<br/>(CoT 4 Bước & Xác Thực Nguồn Gốc ID)"]
        P3["Proxima Ultra<br/>(Programmatic CodeAct & Policy-as-Code)"]
    end

    subgraph SinhDuLieu["2. Động Cơ Sinh & Chuẩn Hóa Dữ Liệu"]
        GEN["sft_generator (Engine Bất Đồng Bộ Đa Luồng)"]
        SAN["scripts/sanitize_dataset.py<br/>(Gắn ID Tool & Chuẩn Hóa Cú Pháp)"]
        D1["carbench_sft_multirole_json.jsonl (1,324 mẫu)"]
        D2["carbench_sft_codeact_python.jsonl (1,443 mẫu)"]
        GEN --> SAN
        SAN --> D1
        SAN --> D2
    end

    subgraph HuanLuyen["3. Tinh Chỉnh SFT & Hợp Nhất Trọng Số"]
        BASE["Mô Hình Gốc: Qwen3-4B-Instruct-2507"]
        LORA["Huấn Luyện LoRA (r=16, alpha=32)<br/>Module: q, k, v, o, gate, up, down_proj<br/>Loss: 0.7222 | Epochs: 2.0 | 148 steps"]
        MERGE["Trọng Số Hợp Nhất 16-Bit (BF16 Safetensors, 7.6GB)"]
        BASE --> LORA --> MERGE
    end

    NenTang --> SinhDuLieu
    D1 --> LORA
    D2 --> LORA

Quy Trình Phục Vụ Suy Luận Khoang Lái Qua vLLM

Bấm để xem mã nguồn biểu đồ Mermaid

sequenceDiagram
    autonumber
    actor TaiXe as Tài Xế / Giám Sát Viên
    participant vLLM as Máy Chủ vLLM (Cổng 8000)
    participant MoHinh as Mô Hình SFT Qwen3-4B Merged
    participant Parser as Bộ Phân Tích Hermes Tool Call
    participant Xe as Hệ Thống Chấp Hành Trên Xe (58 Công Cụ)

    TaiXe->>vLLM: POST /v1/chat/completions (Khẩu lệnh + Danh mục Tools)
    vLLM->>MoHinh: Chuyển Ngữ Cảnh + Chat Template
    Note over MoHinh: Sinh chuỗi suy luận 4 bước:<br/>[Context Audit] -> [Policy Check] -> [Tool Selection] -> [Plan]
    MoHinh-->>Parser: Dòng token hàm gọi
    Parser-->>vLLM: Cấu trúc tool_calls chuẩn hóa
    vLLM-->>TaiXe: Trả về Function Call: get_climate_settings()
    TaiXe->>Xe: Thực thi lệnh get_climate_settings()
    Xe-->>TaiXe: {"status": "success", "temperature": 19.0}
    TaiXe->>vLLM: POST /v1/chat/completions (Kết quả từ cảm biến xe)
    vLLM->>MoHinh: Nạp kết quả tool
    MoHinh-->>vLLM: Phản hồi giọng nói: "Nhiệt độ đã được chỉnh lên 22°C."
    vLLM-->>TaiXe: Câu thoại tự nhiên

Liên Kết Tài Nguyên

Mã Nguồn Pipeline Sinh Dữ Liệu: GitHub: upwitu/carbench_data_generator
Tập Dữ Liệu Huấn Luyện: Hugging Face: upwitu/carbench_sft_winner_dataset
LoRA Adapter Checkpoint: Hugging Face: upwitu/qwen3-4b-sft-all-lora

Downloads last month: 41

Safetensors

Model size

4B params

Tensor type

BF16

upwitu
/

qwen3-4b-sft-all