Instructions to use wesjos/Qwen3.5-4B-HTML-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use wesjos/Qwen3.5-4B-HTML-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="wesjos/Qwen3.5-4B-HTML-GGUF")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("wesjos/Qwen3.5-4B-HTML-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use wesjos/Qwen3.5-4B-HTML-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Use Docker

docker model run hf.co/wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use wesjos/Qwen3.5-4B-HTML-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wesjos/Qwen3.5-4B-HTML-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wesjos/Qwen3.5-4B-HTML-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

SGLang

How to use wesjos/Qwen3.5-4B-HTML-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "wesjos/Qwen3.5-4B-HTML-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wesjos/Qwen3.5-4B-HTML-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "wesjos/Qwen3.5-4B-HTML-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wesjos/Qwen3.5-4B-HTML-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use wesjos/Qwen3.5-4B-HTML-GGUF with Ollama:
```
ollama run hf.co/wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M
```
Unsloth Desktop

How to use wesjos/Qwen3.5-4B-HTML-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use wesjos/Qwen3.5-4B-HTML-GGUF with Docker Model Runner:
```
docker model run hf.co/wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M
```

Lemonade

How to use wesjos/Qwen3.5-4B-HTML-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.5-4B-HTML-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use wesjos/Qwen3.5-4B-HTML-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use wesjos/Qwen3.5-4B-HTML-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "wesjos/Qwen3.5-4B-HTML-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Note

This is a small experiment I conducted to see what happens if I choose to use HTML instead of Markdown in LLM.
there are html tokens in Cot in this model which is low efficent for any task, because models have to waste there energy to think about how to make nice html. ill try to fix that.

🍄 Qwen3.5-4B · HTML (GGUF)

A visual-UI language model fine-tuned from Qwen3.5-4B, quantized to GGUF for llama.cpp. It answers in a small semantic HTML component system — bento grids, metric cards, steppers, callouts — wrapped in one <div class="ai-view"> root, instead of plain Markdown. Reasoning-native, and no system prompt required.

Item	Value
Source repo	`wesjos/Qwen3.5-4B-HTML` — bf16 safetensors + bundled WebUI
Base	`Qwen/Qwen3.5-4B` · 4.66 B (bf16)
Architecture	GGUF `qwen35` · 32 hybrid blocks (24 linear + 8 full attention) + 1 MTP head at `blk.32` · hidden 2560
Context	262,144 native

🖼️ Demo

Copied from the source repo wesjos/Qwen3.5-4B-HTML — these shots are the bf16 checkpoint rendered by the bundled WebUI (webui/, editorial theme), not re-run on these quants. The pink bar at the top of each answer is the collapsed <think> stream; the render below it is the raw output, unmodified.

Code generation

python code: transformer decoder with numpy only

Vision input

describe this picture. — with an image attached

Reasoning

is 9.11 bigger than 9.9

$Reasoning demo: a correction callout, answer cards, a numbered stepper, and a recap grid$

Stepwise arithmetic

8.2x9.12/23=

$Stepwise arithmetic demo: a three-step stepper with code-box derivations and a calculation callout$

Persona

who are you

📦 Files

File	Quant	Size	Tensor mix
`Mario-Qwen3.5-4B-HTML-Q8_0.gguf`	`MOSTLY_Q8_0`	4.29 GiB	257 × Q8_0 + 184 × F32
`Mario-Qwen3.5-4B-HTML-Q4_K_M.gguf`	`MOSTLY_Q4_K_M`	2.59 GiB	222 × Q4_K + 35 × Q6_K + 184 × F32

🚀 Quick Start

# Q4_K_M — fastest, ~2.6 GB on disk
llama-server \
  -hf wesjos/Qwen3.5-4B-HTML-GGUF:Mario-Qwen3.5-4B-HTML-Q4_K_M.gguf \
  --host 0.0.0.0 --port 18085 \
  -ngl 99 -fa on \
  -c 65536

Recommended sampling

Parameter	Value
temperature	1.0
top_p	0.95
top_k	20
min_p	0.0
presence_penalty	1.5
repeat_penalty	1.0

📤 Output Contract

One root element; everything else is an optional child:

Component	Markup
Root	`<div class="ai-view">`
Summary	`<div class="summary-card">` · `<p class="lead">`
Card	`<div class="card">`
Grid	`<div class="grid grid-2">` · `<div class="grid grid-3">`
Metrics	`<div class="metric-group"><div class="metric-card"><span class="metric-label">…</span><span class="metric-value">…</span><span class="metric-sub">…</span></div></div>`
Stepper	`<div class="stepper"><div class="step-item">…</div></div>`
Callout	`<div class="callout callout-info\|warning\|success\|danger"><div class="callout-header">…</div></div>`
Badge	`<span class="badge badge-info\|success\|warning\|danger\|accent">`
Table	`<div class="table-container"><table class="modern-table">…</table></div>`
Feature list	`<div class="feature-list"><div class="feature-item">…</div></div>`
Code block	`<div class="code-box"><div class="code-header"><span class="code-title">…</span></div>…</div>`

Raw sample (message.content verbatim)

<div class="ai-view">
<header class="ai-view-header">
<h1>Good!</h1>
<p class="lead">Everything looks solid from here. Got anything specific you'd like to build or clarify?</p>
</header>
<div class="grid grid-2">
<div class="card">
<span class="badge badge-success">Current state</span>
<h2>The basics check out</h2>
<p>Your message landed cleanly and the conversation is open. This is a strong position to move forward on whatever topic you want to tackle next.</p>
</div>
<div class="card">
<span class="badge badge-info">What's next</span>
<h2>Set the intent</h2>
<p>Tell me what you're trying to accomplish today—whether it's a task, idea, plan, or question—and I'll help you break it down into concrete steps.</p>
</div>
</div>
<div class="metric-group">
<div class="metric-card">
<span class="metric-label">Confidence</span>
<span class="metric-value">High</span>
<span class="metric-sub">Based on your input</span>
</div>
<div class="metric-card">
<span class="metric-label">Priority</span>
<span class="metric-value">Clarify scope</span>
<span class="metric-sub">To make things faster</span>
</div>
</div>
<div class="callout callout-info">
<div class="callout-header">Want something more specific?</div>
<p>Ambiguous prompts work well sometimes, but adding a single sentence about your end goal makes my responses much tighter and more useful.</p>
</div>
</div>

ai-view-header is not in the source repo's theme.css — it renders with default header styling. Fine, but define your own if you want tighter control.