Instructions to use marlalabsAI/Qwen3.5-4B-SX8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use marlalabsAI/Qwen3.5-4B-SX8 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf marlalabsAI/Qwen3.5-4B-SX8
# Run inference directly in the terminal:
llama cli -hf marlalabsAI/Qwen3.5-4B-SX8

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf marlalabsAI/Qwen3.5-4B-SX8
# Run inference directly in the terminal:
llama cli -hf marlalabsAI/Qwen3.5-4B-SX8

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf marlalabsAI/Qwen3.5-4B-SX8
# Run inference directly in the terminal:
./llama-cli -hf marlalabsAI/Qwen3.5-4B-SX8

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf marlalabsAI/Qwen3.5-4B-SX8
# Run inference directly in the terminal:
./build/bin/llama-cli -hf marlalabsAI/Qwen3.5-4B-SX8

Use Docker

docker model run hf.co/marlalabsAI/Qwen3.5-4B-SX8

LM Studio
Jan

vLLM

How to use marlalabsAI/Qwen3.5-4B-SX8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "marlalabsAI/Qwen3.5-4B-SX8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "marlalabsAI/Qwen3.5-4B-SX8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/marlalabsAI/Qwen3.5-4B-SX8

Ollama
How to use marlalabsAI/Qwen3.5-4B-SX8 with Ollama:
```
ollama run hf.co/marlalabsAI/Qwen3.5-4B-SX8
```

Unsloth Studio

How to use marlalabsAI/Qwen3.5-4B-SX8 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for marlalabsAI/Qwen3.5-4B-SX8 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for marlalabsAI/Qwen3.5-4B-SX8 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for marlalabsAI/Qwen3.5-4B-SX8 to start chatting

How to use marlalabsAI/Qwen3.5-4B-SX8 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf marlalabsAI/Qwen3.5-4B-SX8

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "marlalabsAI/Qwen3.5-4B-SX8"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use marlalabsAI/Qwen3.5-4B-SX8 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf marlalabsAI/Qwen3.5-4B-SX8

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "marlalabsAI/Qwen3.5-4B-SX8" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use marlalabsAI/Qwen3.5-4B-SX8 with Docker Model Runner:
```
docker model run hf.co/marlalabsAI/Qwen3.5-4B-SX8
```

Lemonade

How to use marlalabsAI/Qwen3.5-4B-SX8 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull marlalabsAI/Qwen3.5-4B-SX8

Run and chat with the model

lemonade run user.Qwen3.5-4B-SX8-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use marlalabsAI/Qwen3.5-4B-SX8 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf marlalabsAI/Qwen3.5-4B-SX8

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default marlalabsAI/Qwen3.5-4B-SX8

Run Hermes

hermes

Atomic Chat

Qwen3.5-4B — S-X8 v4.3 (7.50 bpp)

Qwen3.5-4B quantized with the S-X8 v4.3 format: 7.50 bits per weight (fully accounted), FP16-level quality and a portable decoder (no tensor cores, no shared memory — runs on any GPU).

Paper (Zenodo): 10.5281/zenodo.21922640 · Repo: github.com/MarlaLabsAI/sx8-quantization

Quality (Qwen3.5-4B, RTX 5060 Ti)

Metric	FP16	S-X8 v4.3	Q8_0
PPL wikitext-2 (PCA runtime)	10.2090	10.2267 (+0.17%)	10.4540 (+2.40%)
Winogrande_s	0.5746	0.5722	0.5746
HellaSwag (0-shot)	0.6965	0.6964	0.6965
ARC-Challenge (0-shot)	0.9172	0.9164	0.9181
MMLU (5-shot)	0.7133	0.7074	0.7087

Size

	S-X8 v4.3	Q8_0
Text file	3.96 GB (−11.6%)	4.48 GB
Complete model (vision+MTP)	4.38 GB (one file)	5.15 GB (GGUF+mmproj)
Weights VRAM	3.955 GB	4.48 GB

Files

Qwen3.5-4B-SX8v43.sx8 (4.38 GB) — native byte-aligned container (byte-exact verifiable)
Qwen3.5-4B-SX8v43.gguf (3.83 GiB) — GGUF with native type GGML_TYPE_SX8 (requires the llama.cpp fork with S-X8 support)

How to use the model

Option A — GGUF with the llama.cpp fork (recommended):

Get the patch: github.com/MarlaLabsAI/sx8-quantization → llama-cpp-sx8.patch
Clone llama.cpp at commit 7c203670f and apply: git apply llama-cpp-sx8.patch
Build with CUDA (-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=120)
Run:
```
./build/bin/llama-cli -m Qwen3.5-4B-SX8v43.gguf -ngl 99 -p "Hello, how are you?"
```
Measured decode: 63.79 tok/s on RTX 5060 Ti (faster than Q8_0 in real-world use).

Option B — .sx8v43 container with the S-X8 runtime (developers/research):

Clone github.com/MarlaLabsAI/sx8-quantization (scripts: eval_common.py, sx8_container_v43.py, kernels)
Load: from eval_common import load_model; m, tok, meta = load_model(quantized=True, mode="v43", source_file="Qwen3.5-4B-SX8v43.sx8")
The container is byte-aligned and byte-exact verifiable (381/381 tensors).

Links: GitHub repo · Paper (Zenodo, DOI)

How to cite

Vidal Leandro, M. (2026). S-X8 v4.3: A 7.50-Bits-Per-Weight Quantization Format with FP16-Level Quality and Portable Decoding. Zenodo. https://doi.org/10.5281/zenodo.21922640

Usage files in this repo (self-contained)

llama-cpp-sx8.patch — llama.cpp fork patch (apply on commit 7c203670f)
SX8_FLASH_V4_3_SPEC.md · SX8_FLASH_V4_3_CONTAINER.md — format and container specs
S-X-METHODOLOGY.md — methodology authorship
eval_common.py · kernel_sx8_v43.py · kernel_sx8_v4.py · sx8_container_v43.py — runtime for the .sx8v43
paper-sx8.pdf · paper-sx8-ES.pdf — the paper (EN/ES)

Transparency

The conceptual seeds of S-X8 come from an independent mathematical analysis of the image of the Shroud of Turin; the full study is published in the project repo with its own re-verification: github.com/MarlaLabsAI/sx8-quantization → shroud-turin-study/. See the paper's Appendix A and docs/IDEA-PROVENANCE.md. The format itself is validated empirically in this repository; the study is disclosed only as the source of inspiration.

License and attribution

Apache-2.0. The S-X8 format, kernels and scripts in this repository are original work (Copyright (C) 2026 Martí Vidal Leandro), not a Qwen work. The quantized model uses Qwen3.5-4B by the Qwen Team (Alibaba Group), Apache-2.0 (model card) as the base model; only the weights were quantized to the S-X8 v4.3 format (see NOTICE), no other modifications.

Qwen3.5-4B — S-X8 v4.3 (7,50 bpp) [ES]

Qwen3.5-4B cuantizado con el formato S-X8 v4.3: 7,50 bits por peso (contabilizados al completo), calidad de nivel FP16 y decodificador portable (sin tensor cores, sin memoria compartida — funciona en cualquier GPU).

Paper (Zenodo): 10.5281/zenodo.21922640 · Repo: github.com/MarlaLabsAI/sx8-quantization

Calidad (Qwen3.5-4B, RTX 5060 Ti)

Métrica	FP16	S-X8 v4.3	Q8_0
PPL wikitext-2 (runtime PCA)	10,2090	10,2267 (+0,17%)	10,4540 (+2,40%)
Winogrande_s	0,5746	0,5722	0,5746
HellaSwag (0-shot)	0,6965	0,6964	0,6965
ARC-Challenge (0-shot)	0,9172	0,9164	0,9181
MMLU (5-shot)	0,7133	0,7074	0,7087

Tamaño

	S-X8 v4.3	Q8_0
Archivo de texto	3,96 GB (−11,6%)	4,48 GB
Modelo completo (visión+MTP)	4,38 GB (un archivo)	5,15 GB (GGUF+mmproj)
VRAM pesos	3,955 GB	4,48 GB

Ficheros

Qwen3.5-4B-SX8v43.sx8 (4,38 GB) — contenedor nativo byte-aligned (verificable byte-exacto)
Qwen3.5-4B-SX8v43.gguf (3,83 GiB) — GGUF con tipo nativo GGML_TYPE_SX8 (requiere el fork de llama.cpp con soporte S-X8)

Cómo usar el modelo

Opción A — GGUF con el fork de llama.cpp (recomendado):

Descarga el patch: github.com/MarlaLabsAI/sx8-quantization → llama-cpp-sx8.patch
Clona llama.cpp en el commit 7c203670f y aplica: git apply llama-cpp-sx8.patch
Compila con CUDA (-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=120)
Ejecuta:
```
./build/bin/llama-cli -m Qwen3.5-4B-SX8v43.gguf -ngl 99 -p "Hola, ¿cómo estás?"
```
Decode medido: 63,79 tok/s en RTX 5060 Ti (más rápido que Q8_0 en uso real).

Opción B — Contenedor .sx8v43 con el runtime S-X8 (desarrolladores/investigación):

Clona github.com/MarlaLabsAI/sx8-quantization (scripts: eval_common.py, sx8_container_v43.py, kernels)
Carga: from eval_common import load_model; m, tok, meta = load_model(quantized=True, mode="v43", source_file="Qwen3.5-4B-SX8v43.sx8")
El contenedor es byte-aligned y verificable byte-exacto (381/381 tensores).

Enlaces: Repo GitHub · Paper (Zenodo, DOI)

Cómo citar

Vidal Leandro, M. (2026). S-X8 v4.3: Un formato de cuantización a 7,50 bits por peso con calidad de FP16 y decodificación portable. Zenodo. https://doi.org/10.5281/zenodo.21922640

Archivos de uso en este repo (autosuficiente)

llama-cpp-sx8.patch — patch del fork de llama.cpp (aplicar sobre el commit 7c203670f)
SX8_FLASH_V4_3_SPEC.md · SX8_FLASH_V4_3_CONTAINER.md — spec del formato y del contenedor
S-X-METHODOLOGY.md — autoría de la metodología
eval_common.py · kernel_sx8_v43.py · kernel_sx8_v4.py · sx8_container_v43.py — runtime para el .sx8v43
paper-sx8.pdf · paper-sx8-ES.pdf — el paper (EN/ES)

Transparencia

Las semillas conceptuales de S-X8 provienen de un análisis matemático independiente de la imagen de la Sábana Santa de Turín; el estudio completo se publica en el repo del proyecto con su propia re-verificación: github.com/MarlaLabsAI/sx8-quantization → shroud-turin-study/. Ver el Apéndice A del paper y docs/IDEA-PROVENANCE.md del repo. El formato en sí está validado empíricamente en este repositorio; el estudio se menciona solo como fuente de inspiración.

Licencia y atribución

Apache-2.0. El formato S-X8, los kernels y los scripts de este repositorio son obra original (Copyright (C) 2026 Martí Vidal Leandro), no un trabajo de Qwen. El modelo cuantizado usa Qwen3.5-4B de Qwen Team (Alibaba Group), Apache-2.0 (model card) como modelo base; solo se cuantizaron los pesos al formato S-X8 v4.3 (ver NOTICE), sin otras modificaciones.

Downloads last month: 840

GGUF

Model size

4B params

Architecture

qwen35

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for marlalabsAI/Qwen3.5-4B-SX8

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Quantized

(355)

this model