Instructions to use webmp3/hfqa-auto-proof-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use webmp3/hfqa-auto-proof-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/hfqa-auto-proof-GGUF:Q2_K
# Run inference directly in the terminal:
llama cli -hf webmp3/hfqa-auto-proof-GGUF:Q2_K

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/hfqa-auto-proof-GGUF:Q2_K
# Run inference directly in the terminal:
llama cli -hf webmp3/hfqa-auto-proof-GGUF:Q2_K

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf webmp3/hfqa-auto-proof-GGUF:Q2_K
# Run inference directly in the terminal:
./llama-cli -hf webmp3/hfqa-auto-proof-GGUF:Q2_K

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf webmp3/hfqa-auto-proof-GGUF:Q2_K
# Run inference directly in the terminal:
./build/bin/llama-cli -hf webmp3/hfqa-auto-proof-GGUF:Q2_K

Use Docker

docker model run hf.co/webmp3/hfqa-auto-proof-GGUF:Q2_K

Llama new
LM Studio
Jan
Ollama
How to use webmp3/hfqa-auto-proof-GGUF with Ollama:
```
ollama run hf.co/webmp3/hfqa-auto-proof-GGUF:Q2_K
```
Unsloth Desktop

How to use webmp3/hfqa-auto-proof-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/hfqa-auto-proof-GGUF:Q2_K

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "webmp3/hfqa-auto-proof-GGUF:Q2_K"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use webmp3/hfqa-auto-proof-GGUF with Docker Model Runner:
```
docker model run hf.co/webmp3/hfqa-auto-proof-GGUF:Q2_K
```

Lemonade

How to use webmp3/hfqa-auto-proof-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull webmp3/hfqa-auto-proof-GGUF:Q2_K

Run and chat with the model

lemonade run user.hfqa-auto-proof-GGUF-Q2_K

List all available models

lemonade list

Hermes Agent

How to use webmp3/hfqa-auto-proof-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/hfqa-auto-proof-GGUF:Q2_K

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default webmp3/hfqa-auto-proof-GGUF:Q2_K

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use webmp3/hfqa-auto-proof-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/hfqa-auto-proof-GGUF:Q2_K

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "webmp3/hfqa-auto-proof-GGUF:Q2_K" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GGUF quantization of Qwen/Qwen2.5-0.5B-Instruct

Independent community quantization. Not an official release of the upstream author.

1. Model Overview

This repository is produced by the local HF Quant Agent control plane. It records only checks that actually ran. Unmeasured quality, speed and compatibility are marked untested.

2. Original Model / Attribution

Model/checkpoint identity: Qwen/Qwen2.5-0.5B-Instruct
Submitted repository: Qwen/Qwen2.5-0.5B-Instruct
Submitted commit: 7ae557604adf67be50417f59c2c2f167def9a775
Declared base models: Qwen/Qwen2.5-0.5B
Declared relation: finetune
Declared base-model commit revisions: not resolved by this metadata; quantization-input commits are recorded separately
Attribution evidence: pinned model card
Assignment comes from metadata. A name match or a model-card declaration is not a runtime/quality test.
Source: Qwen/Qwen2.5-0.5B-Instruct
Publisher of this quantization: webmp3, when a release is actually uploaded
Upstream authors retain their license and attribution
GGUF source publisher: Qwen/Qwen2.5-0.5B-Instruct-GGUF
GGUF source commit: 9217f5db79a29953eb74d5343926648285ec7e67
GGUF source filename: qwen2.5-0.5b-instruct-q8_0.gguf
GGUF source SHA-256: ca59ca7f13d0e15a8cfa77bd17e65d24f6844b554a7b6c12e07a5f89ff76844e
GGUF source provenance verified: True

3. Source Revision

Pinned revision: 7ae557604adf67be50417f59c2c2f167def9a775

Jobs are bound to this SHA. A later upstream commit does not silently change a running or completed job.

4. Supported GGUF Quantizations

Requested: Q2_K, Q4_K_M

File	Bytes	SHA-256	Quant
`qwen2.5-0.5b-instruct-q8_0-Q2_K-requant.gguf`	415182688	`704126881e1c6da2be5d9d04deb925d6945c473930c6dd355743ece04612fd55`	Q2_K
`qwen2.5-0.5b-instruct-q8_0-Q4_K_M-requant.gguf`	491400032	`a7fcafef1fb79acf1dd006bbae5ceef2d9068a9b90510c8aa384b3d021df10aa`	Q4_K_M

5. Quantization Methodology

Route: native_gguf_requant

Adapter: standard-llamacpp

This route performs existing-GGUF requantization from the pinned Q8_0/F16/BF16 source GGUF identified above. This job did not convert the upstream BF16/safetensors checkpoint. Source tensor storage measured in the GGUF: F32: 121, Q8_0: 170.

No importance-matrix calibration was run by this job. Requantizing a previously quantized source can add quantization error. Estimated sizes are planning ratios, not measurements, unless a file byte count is listed above.

The target quant label describes the selected stock quantizer preset. Stock llama.cpp may retain floating-point tensors or use fallback tensor types; every tensor is not necessarily stored in the named target type. The table records measured output tensor types and only reports a fallback count when the quantizer printed one.

Source tensor count: 291. Source full tensor-name set SHA-256: 70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0. New and reused outputs must retain the source tensor count and complete name set.

File	Measured tensor-type counts	Quantizer-reported fallback tensors	Full tensor-name set SHA-256
`qwen2.5-0.5b-instruct-q8_0-Q2_K-requant.gguf`	F32: 121, Q3_K: 24, Q4_0: 121, Q5_0: 24, Q8_0: 1	145	`70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0`
`qwen2.5-0.5b-instruct-q8_0-Q4_K_M-requant.gguf`	F32: 121, Q4_K: 12, Q5_0: 133, Q6_K: 12, Q8_0: 13	145	`70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0`

6. Architecture Information

model_type / architecture: qwen2
parameters: 494032768
experts: n/a or not declared
source formats observed: safetensors, bfloat16
custom code present: False
trust_remote_code executed: False

7. Conversion and Runtime Compatibility

GGUF structure valid: True
Runtime load tested: True
Inference tested: True

A structurally valid GGUF is not automatically a working model.

File	Runtime build reported by executable	Runtime executable SHA-256
`qwen2.5-0.5b-instruct-q8_0-Q2_K-requant.gguf`	`b11669-620d2225c`	`486568ae9dd8682e160861a129b70f4a5efa73191542ab0cf80674b8b6d4f814`
`qwen2.5-0.5b-instruct-q8_0-Q4_K_M-requant.gguf`	`b11669-620d2225c`	`486568ae9dd8682e160861a129b70f4a5efa73191542ab0cf80674b8b6d4f814`

8. Download and Usage Instructions

Use a llama.cpp build that supports this architecture. Example, after you confirm compatibility:

llama-cli -m <file>.gguf -p "Hello" -n 32

This card does not claim that every runtime can load the file.

9. Validation Results

All 2 requested quants passed structure and one real CPU inference each. No quality benchmark was run.

10. Limitations

No coding benchmark, perplexity or quality score is claimed unless a measured value is copied from this job's own log.
Shard resume and upload verification apply only to artifacts listed in this card.
Oracle DNS services take priority. Qualified large GGUF processing uses bounded streaming; a release still requires actual runtime validation of its exact artifacts.

11. License Information

Source license: apache-2.0

Decision: apache-2.0 allows redistribution with attribution

If the license does not allow redistribution, this card must not be uploaded.

12. Reproduction Information

Control plane: HF Quant Agent
Planner route: native_gguf_requant
Workflow: none
Blockers: none recorded

中文说明 · 樱花 Sakura（点击展开）

1. 概述

本仓库由本地 HF Quant Agent 控制平面生成，只记录实际运行过的检查。未测量的质量、速度与兼容性均标记为“未测试”。

独立的社区量化版本，并非上游作者的官方发布。

2. 原始模型与署名

模型/检查点：Qwen/Qwen2.5-0.5B-Instruct
提交的仓库：Qwen/Qwen2.5-0.5B-Instruct
提交的版本：7ae557604adf67be50417f59c2c2f167def9a775
声明的基础模型：Qwen/Qwen2.5-0.5B
声明的关系：finetune
署名依据：固定版本的模型卡
归属信息来自元数据；名称相符或模型卡声明并不代表经过运行或质量测试。
来源：Qwen/Qwen2.5-0.5B-Instruct
本量化的发布者：webmp3
上游作者保留其许可证与署名
GGUF 来源发布者：Qwen/Qwen2.5-0.5B-Instruct-GGUF
GGUF 来源提交：9217f5db79a29953eb74d5343926648285ec7e67
GGUF 来源文件名：qwen2.5-0.5b-instruct-q8_0.gguf
GGUF 来源 SHA-256：ca59ca7f13d0e15a8cfa77bd17e65d24f6844b554a7b6c12e07a5f89ff76844e
GGUF 来源出处已验证：True

3. 源版本

固定版本：7ae557604adf67be50417f59c2c2f167def9a775

任务绑定到此 SHA。上游之后的新提交不会悄悄改变正在运行或已完成的任务。

4. 已提供的 GGUF 量化

请求的量化：Q2_K, Q4_K_M

文件	字节数	SHA-256	量化
`qwen2.5-0.5b-instruct-q8_0-Q2_K-requant.gguf`	415182688	`704126881e1c6da2be5d9d04deb925d6945c473930c6dd355743ece04612fd55`	Q2_K
`qwen2.5-0.5b-instruct-q8_0-Q4_K_M-requant.gguf`	491400032	`a7fcafef1fb79acf1dd006bbae5ceef2d9068a9b90510c8aa384b3d021df10aa`	Q4_K_M

5. 量化方法

路线：native_gguf_requant

适配器：standard-llamacpp

该路线从上方标明的固定版本 Q8_0/F16/BF16 源 GGUF 进行现有 GGUF 的再量化。本任务没有转换上游的 BF16/safetensors 检查点。源 GGUF 中测得的张量存储：F32: 121, Q8_0: 170。

本任务没有运行重要性矩阵（imatrix）校准。对已量化的源再次量化可能增加量化误差。除非上表列出了文件字节数，否则大小估计只是规划比例，不是实测值。

目标量化标签表示所选的 llama.cpp 标准量化预设。标准 llama.cpp 可能保留浮点张量或使用回退张量类型，因此并非每个张量都以所标注的目标类型存储。下表记录实测的输出张量类型，只有当量化器打印了回退数量时才会给出该数量。

源张量数量：291。源张量名称全集 SHA-256：70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0。新生成与复用的输出必须保持源张量数量和完整名称集合不变。

文件	实测张量类型计数	量化器报告的回退张量	张量名称全集 SHA-256
`qwen2.5-0.5b-instruct-q8_0-Q2_K-requant.gguf`	F32: 121, Q3_K: 24, Q4_0: 121, Q5_0: 24, Q8_0: 1	145	`70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0`
`qwen2.5-0.5b-instruct-q8_0-Q4_K_M-requant.gguf`	F32: 121, Q4_K: 12, Q5_0: 133, Q6_K: 12, Q8_0: 13	145	`70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0`

6. 架构信息

model_type / 架构：qwen2
参数量：494032768
专家数：n/a or not declared
观察到的源格式：safetensors, bfloat16
含自定义代码：False
已执行 trust_remote_code：False

7. 转换与运行时兼容性

GGUF 结构有效：True
已测试运行时加载：True
已测试推理：True

结构有效的 GGUF 不一定是可用的模型。

文件	可执行程序报告的运行时版本	运行时可执行文件 SHA-256
`qwen2.5-0.5b-instruct-q8_0-Q2_K-requant.gguf`	`b11669-620d2225c`	`486568ae9dd8682e160861a129b70f4a5efa73191542ab0cf80674b8b6d4f814`
`qwen2.5-0.5b-instruct-q8_0-Q4_K_M-requant.gguf`	`b11669-620d2225c`	`486568ae9dd8682e160861a129b70f4a5efa73191542ab0cf80674b8b6d4f814`

8. 下载与使用

请使用支持该架构的 llama.cpp 版本。确认兼容后的示例：

llama-cli -m <file>.gguf -p "Hello" -n 32

本说明不声称每个运行时都能加载这些文件。

9. 验证结果

All 2 requested quants passed structure and one real CPU inference each. No quality benchmark was run.（以上为英文原文）

10. 局限

除非数值取自本任务自己的日志，否则不声称任何编程基准、困惑度或质量分数。
分片续传与上传校验仅适用于本说明中列出的文件。
Oracle 的 DNS 服务优先。已验证的大型 GGUF 采用有界流式处理；正式发布仍需对其确切文件进行实际运行时验证。

11. 许可证信息

源许可证：apache-2.0

判定（英文原文）：apache-2.0 allows redistribution with attribution

如果许可证不允许再分发，则不得上传本说明。

12. 复现信息

控制平面：HF Quant Agent
规划路线：native_gguf_requant
工作流：none
阻塞项：无记录

完整独立的中文版：README_zh.md

Downloads last month: -

GGUF

Model size

0.6B params

Architecture

qwen2

Hardware compatibility

2-bit

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for webmp3/hfqa-auto-proof-GGUF

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-0.5B-Instruct

Quantized

(353)

this model