Instructions to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp
# Run inference directly in the terminal:
llama cli -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp
# Run inference directly in the terminal:
llama cli -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp
# Run inference directly in the terminal:
./build/bin/llama cli -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Use Docker

docker model run hf.co/Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Llama new
LM Studio
Jan

vLLM

How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Ollama
How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with Ollama:
```
ollama run hf.co/Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp
```
Unsloth Desktop

How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with Docker Model Runner:
```
docker model run hf.co/Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp
```

Lemonade

How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Run and chat with the model

lemonade run user.Qwen3.5-4B-3D-Coder-ik_llama.cpp-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Austriani // ik_llama.cpp // GGUF

Qwen3.5-4B-3D-Coder

Holographic Quantization Archive for Three.js and Code

Parameters

Layers

1024

Imatrix Ctx

500

Cycles

Base Architecture

This repository hosts ik_llama.cpp-compatible GGUF quantizations of ccore/Qwen3.5-4B-3D-Coder. This is an independent fine-tune of Qwen/Qwen3.5-4B by CCORE, optimized for general-purpose coding and high-quality 3D scene generation. It writes self-contained Three.js HTML and JavaScript scenes directly.

The architecture is a dense 4B parameter hybrid model. It uses a repeating layout of 3 Gated DeltaNet (linear attention) layers followed by 1 gated full-attention layer, stacked 8 times. The model features a hidden size of 2560 and a native 262,144-token context window (extendable to 1M with YaRN). No abliteration or behavioral modification has been applied.

Note on MTP Layer: The Multi-Token Prediction (MTP) layer has been removed from these quantizations, as the current quantization pipeline (ik_llama.cpp) does not support it. However, if you have the MTP weights for Qwen3.5-4B installed and enabled in your inference setup, this layer can be manually added back to restore full speculative decoding capabilities.

MaxOptima Custom Quant

This release includes MaxOptima, a bespoke, surgical quantization mix designed specifically for the 3:1 hybrid attention architecture of Qwen3.5. It targets the full-attention blocks and boundary layers for high precision, while compressing the FFNs to save space.

MaxOptima Strategy: Full-attention layers (blocks 3, 7, 11, 15, 19, 23, 27, 31) are kept at Q5_K to preserve global context and logic. The embeddings, output layer, and boundary blocks (0 and 31) are kept at Q5_K and Q6_K. The attention gates use IQ4_KSS, while the bulk FFN up/down are compressed to IQ4_KSS and the FFN gates to IQ4_KS. This results in a file significantly smaller than a standard Q4_K_M, but with the attention precision required for complex 3D math.

View Full Custom Quant Configuration

--custom-q "^output\.weight=Q6_K,token_embd\.weight=Q5_K,ssm_out\.weight=Q5_K,attn_qkv\.weight=Q5_K,attn_gate\.weight=IQ4_KSS,blk\.(3|7|11|15|19|23|27|31)\.attn_q\.weight=Q5_K,blk\.(3|7|11|15|19|23|27|31)\.attn_k\.weight=Q5_K,blk\.(3|7|11|15|19|23|27|31)\.attn_v\.weight=Q5_K,blk\.(3|7|11|15|19|23|27|31)\.attn_output\.weight=Q5_K,blk\.(0|31)\.ffn_down\.weight=Q5_K,blk\.(0|31)\.ffn_up\.weight=Q5_K,ffn_down\.weight=IQ4_KSS,ffn_up\.weight=IQ4_KSS,ffn_gate\.weight=IQ4_KS,.*=IQ4_KS"

Available Quants

MaxOptima // Custom Hybrid Mix

MaxOptima

Custom mix. Full-attention blocks at Q5_K, FFNs at IQ4_KSS/IQ4_KS. Best quality-to-size ratio for 3D coding.

Standard IQK // High & Mid Fidelity

IQ5_K

Higher fidelity 5-bit. Best quality-to-size for systems with sufficient memory.

IQ4_K

Standard 4-bit IQK. Uses 256-value super-blocks for scaling.

IQ4_KS

"Small" 4-bit variant with tighter packing. Lower memory footprint.

IQ4_KS_R4

Row-interleaved variant of IQ4_KS, optimized for ik_llama.cpp's matmul kernels.

Trellis // Alternative 4-Bit

IQ4_KT

Dynamic programming trellis quant. Uses a fundamentally different error-minimization strategy than IQK.

Imatrix Specification

This release utilizes the exact same V3 Calibration Dataset used across previous Austriani releases. No datasets were changed. The calibration was executed at 500 cycles per chunk with a 1024-token context window, processing approximately 1000 chunks.

Compatibility

These GGUF files are built exclusively for ik_llama.cpp. IQK, Trellis, and MaxOptima quants will not load in mainline llama.cpp. Please refer to the official base repository for mainline GGUF compatibility.

Downloads last month: 148

GGUF

Model size

4B params

Architecture

qwen35

Hardware compatibility

Model tree for Austriani/Qwen3.5-4B-3D-Coder-ik_llama.cpp

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

ccore/Qwen3.5-4B-3D-Coder

Quantized

(14)

this model