Instructions to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Use Docker

docker model run hf.co/ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ngquocvinh/GLM-4.7-Flash-Coder-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ngquocvinh/GLM-4.7-Flash-Coder-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Ollama
How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with Ollama:
```
ollama run hf.co/ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M
```
Unsloth Desktop

How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with Docker Model Runner:
```
docker model run hf.co/ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M
```

Lemonade

How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.GLM-4.7-Flash-Coder-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ngquocvinh/GLM-4.7-Flash-Coder-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ngquocvinh/GLM-4.7-Flash-Coder-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-4.7-Flash-Coder GGUF

Community GGUF quantizations of whitecircle/GLM-4.7-Flash-Coder.

☕ If this GGUF made your day easier, a coffee would make mine.
Send a coffee ☕
I build and test these releases myself. Your coffee helps keep me going.
Thank you for supporting this work.

About GLM-4.7-Flash-Coder

GLM-4.7-Flash-Coder is an agentic coding fine-tune of zai-org/GLM-4.7-Flash. The upstream model card describes a 30B Mixture-of-Experts model trained on software-engineering trajectories, with tool use, reasoning, and concurrent tool-call behavior. The upstream training configuration uses a 75k-token sequence length; its model configuration declares a 202,752-token maximum position limit. This release does not claim long-context behavior beyond the runtime validation documented below.

The model uses the GLM-4.7 Flash chat template and XML-style tool calls. See the official upstream model card for the original model, training, and evaluation details.

This repository contains quantizations only. No training or fine-tuning was performed here.

Fidelity measurements

These are next-token fidelity measurements against the locked BF16 GGUF, not a task benchmark. Every published quantization was evaluated on the separate wiki.valid.raw hold-out from WikiText-2, using 8 sequential chunks at context 2048 with llama-perplexity from llama.cpp commit 4a3635c32fc9f044c2bde9ebeabf50c7e1ec5991. The BF16 reference used partial GPU 0 offload (-ngl 10); quantized files used GPU 0 with -ngl 20 for the larger files and -ngl 99 for the smaller files. Values are aggregate means across the 8 evaluated chunks. The calibration text used for the imatrix was kept separate from this hold-out.

Lower KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate behavior closer to BF16. The two bold rows are measured memory/fidelity candidates based on file size and hold-out fidelity; no precise VRAM claim is made here. Quality can vary by workload, prompt, and runtime settings.

File	Size (GB)	Mean KLD ↓	Top-1 vs BF16 ↑	ΔPPL	RMS Δp
`GLM-4.7-Flash-Coder-Q8_0.gguf`	31.842800	0.042086	95.760%	-0.142%	4.503%
`GLM-4.7-Flash-Coder-Q6_K.gguf`	24.614787	0.059268	93.707%	+2.321%	5.816%
`GLM-4.7-Flash-Coder-Q5_K_M.gguf`	21.264516	0.070494	92.290%	+1.830%	6.328%
`GLM-4.7-Flash-Coder-Q5_K_S.gguf`	20.664288	0.082405	91.789%	+1.575%	6.945%
`GLM-4.7-Flash-Coder-Q4_K_M.gguf`	18.132722	0.131201	88.087%	+1.137%	9.298%
`GLM-4.7-Flash-Coder-Q4_K_S.gguf`	17.097991	0.127925	87.402%	+1.974%	9.307%
`GLM-4.7-Flash-Coder-IQ4_XS.gguf`	16.196309	0.147614	87.671%	+3.496%	9.835%
`GLM-4.7-Flash-Coder-Q3_K_L.gguf`	15.591780	0.246380	82.160%	+3.192%	13.084%
`GLM-4.7-Flash-Coder-Q3_K_M.gguf`	14.405579	0.243714	81.989%	+2.300%	13.333%
`GLM-4.7-Flash-Coder-IQ3_M.gguf`	13.238423	0.327159	78.715%	+14.599%	15.956%
`GLM-4.7-Flash-Coder-Q3_K_S.gguf`	13.034401	0.313705	79.790%	+4.298%	15.357%
`GLM-4.7-Flash-Coder-Q2_K.gguf`	11.042966	0.865406	65.579%	+72.007%	26.452%
`GLM-4.7-Flash-Coder-IQ2_XS.gguf`	8.877017	1.673547	51.381%	+267.260%	37.874%
`GLM-4.7-Flash-Coder-IQ1_M.gguf`	6.886802	8.988249	3.739%	+506816.668%	65.582%

See the compact quality summary for machine-readable values and the reproducibility manifest for corpus hashes and the exact evaluation profile.

Quick start

The first validated artifact is GLM-4.7-Flash-Coder-Q8_0.gguf; run it with llama.cpp:

./llama-cli \
  -m GLM-4.7-Flash-Coder-Q8_0.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

The included chat_template.jinja should be used for chat and tool-call serialization. Tool calling and reasoning claims are limited to the validation profiles recorded in the reproducibility manifest.

Reproducibility and validation

Artifacts are being released incrementally. Each quantization is generated directly from the locked BF16 GGUF source, validated with the local llama.cpp runtime, checksummed, and uploaded in a separate Hub commit. The baseline ladder is:

Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M, Q2_K, IQ2_XS, IQ1_M, Q1_0.

Additional direct-from-BF16 profile variants released here are Q5_K_S, Q4_K_S, IQ4_XS, Q3_K_L, Q3_K_S, and IQ3_M.

The reproducibility manifest records the upstream revision, source hashes, converter/runtime revision, calibration input, quantization commands, validation profile, and publication status. The model-specific calibration text and importance matrix are included under reproducibility/. The public SHA256SUMS.txt records checksums for published files. Raw conversion, calibration, quantization, smoke-test, and benchmark logs remain local under reports/.

Q8_0, Q6_K, Q5_K_M, and Q4_K_M have passed CPU and GPU 0 load/generate smoke tests with the included chat template. Q5_K_S passed CPU smoke and a GPU 0 partial-offload smoke at -ngl 20 with context 2048; a full-offload attempt exceeded the available GPU 0 memory and is not claimed as a full-GPU profile. Q4_K_S, IQ4_XS, Q3_K_L, Q3_K_S, and IQ3_M passed CPU and full GPU 0 smoke at -ngl 99 with context 2048. Q3_K_M, Q2_K, IQ2_XS, and IQ1_M have passed CPU smoke testing and are published; GPU 0 smoke was not run for these lower formats. Q1_0 was attempted but is omitted because this runtime has no fallback for a non-block-aligned tensor; no invalid artifact is published.

The fidelity table above compares every published quantization with the same BF16 reference. Raw conversion, fidelity, quantization, smoke-test, and benchmark logs remain local under reports/; the public package contains only the compact quality summary and reproducibility metadata needed to reproduce the release.

License and attribution

The upstream model card declares the MIT license; see LICENSE and the upstream model card.

These are community GGUF quantizations, not an official whitecircle/GLM-4.7-Flash-Coder release or endorsement.

Downloads last month: 4,607

GGUF

Model size

30B params

Architecture

deepseek2

Hardware compatibility

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

View +1 variant

Model tree for ngquocvinh/GLM-4.7-Flash-Coder-GGUF

Base model

zai-org/GLM-4.7-Flash

Finetuned

whitecircle/GLM-4.7-Flash-Coder

Quantized

(3)

this model

ngquocvinh
/

GLM-4.7-Flash-Coder-GGUF

GLM-4.7-Flash-Coder GGUF

About GLM-4.7-Flash-Coder

Fidelity measurements

Quick start

Reproducibility and validation

License and attribution

Model tree for ngquocvinh/GLM-4.7-Flash-Coder-GGUF

Space using ngquocvinh/GLM-4.7-Flash-Coder-GGUF 1