Instructions to use agentscope-ai/Piccolo-9B-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use agentscope-ai/Piccolo-9B-BF16 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf agentscope-ai/Piccolo-9B-BF16:BF16
# Run inference directly in the terminal:
llama cli -hf agentscope-ai/Piccolo-9B-BF16:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf agentscope-ai/Piccolo-9B-BF16:BF16
# Run inference directly in the terminal:
llama cli -hf agentscope-ai/Piccolo-9B-BF16:BF16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf agentscope-ai/Piccolo-9B-BF16:BF16
# Run inference directly in the terminal:
./build/bin/llama cli -hf agentscope-ai/Piccolo-9B-BF16:BF16

Use Docker

docker model run hf.co/agentscope-ai/Piccolo-9B-BF16:BF16

Llama new
LM Studio
Jan
Ollama
How to use agentscope-ai/Piccolo-9B-BF16 with Ollama:
```
ollama run hf.co/agentscope-ai/Piccolo-9B-BF16:BF16
```
Unsloth Desktop

How to use agentscope-ai/Piccolo-9B-BF16 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf agentscope-ai/Piccolo-9B-BF16:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "agentscope-ai/Piccolo-9B-BF16:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use agentscope-ai/Piccolo-9B-BF16 with Docker Model Runner:
```
docker model run hf.co/agentscope-ai/Piccolo-9B-BF16:BF16
```

Lemonade

How to use agentscope-ai/Piccolo-9B-BF16 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull agentscope-ai/Piccolo-9B-BF16:BF16

Run and chat with the model

lemonade run user.Piccolo-9B-BF16-BF16

List all available models

lemonade list

Hermes Agent

How to use agentscope-ai/Piccolo-9B-BF16 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf agentscope-ai/Piccolo-9B-BF16:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default agentscope-ai/Piccolo-9B-BF16:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use agentscope-ai/Piccolo-9B-BF16 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf agentscope-ai/Piccolo-9B-BF16:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "agentscope-ai/Piccolo-9B-BF16:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Piccolo-9B-BF16

GitHub · 🌐 Decision · Hugging Face · ModelScope · coming soon

Unquantized BF16 text GGUF converted directly from the complete Piccolo-9B Safetensors checkpoint. Piccolo provides candidate-token decisions for tool-operation review and context retention. See the project and Decision Playground.

Precision and conversion

This release uses convert_hf_to_gguf.py --outtype bf16 --no-mtp; it does not pass through Q8/Q4 or any other integer quantization. The converter retains 250 tensors in BF16 and 177 tensors in F32 (including normalization and recurrent-state-related tensors). BF16 is the checkpoint's main weight precision; this release does not claim equivalence to FP32 computation. Text backbone/output tensors are included. Auxiliary vision/MTP tensors remain available in the complete original Safetensors repository and are outside this text-only GGUF.

The 4 standard GGUF shards total 17.921 GB. Download all shards into the same directory, then load Piccolo-9B-BF16-00001-of-00004.gguf. Each shard is below 5 GB; the complete model, KV/state buffers and runtime workspace require additional memory. release-manifest.json lists byte sizes and SHA-256 checksums. conversion-provenance.json identifies the exact base revision, original source file hashes, converter, parent GGUF and tensor precisions. Sharding only partitions existing tensors; it does not quantize them.

hf download agentscope-ai/Piccolo-9B-BF16 --local-dir Piccolo-9B-BF16

Required decision inference

Use the Piccolo-aware llama.cpp adaptation described in QwenPaw PR #8463 and local inference documentation. This model requires bidirectional prompt attention in full-attention layers, unchanged linear attention, the exact flex_twice layout and candidate-token probability readout at temperature 1. Generate no answer tokens. Ordinary causal chat completion and unpatched runtimes do not implement this protocol. The serialized decision type remains blink, the runtime format identifier used by these weights; the release and repository name is Piccolo.

The evaluated CUDA configuration uses one slot and context/batch/ubatch 8192, with prefix caching and chunked prefill disabled. Avoid prompt truncation. The source checkpoint's configured maximum context is not a claim that this release has been validated at that length. Confirm that the QwenPaw version you use recognizes the -BF16 repository as a decision model before selecting it; publishing these weights does not automatically add a model entry or extend the existing application's repository allowlist.

Numerical evidence and limits

This exact parent BF16 artifact was used as the reference for the expanded 265-request / 556-judgment synthetic regression, covering review, context retention and scoring. Its four-question check against full-checkpoint inference had maximum absolute candidate probability difference 0.001873. The complete-checkpoint check is small and is not a benchmark score, broad accuracy measurement, risk calibration or safety certification. validation.json records the measured result and inference identity. The expanded regression uses this BF16 artifact as its baseline; it does not establish independent ground-truth task correctness.

Release entry points

The model matrix lists all 12 published full/BF16/Q8_0/Q4_K_M repositories. Full-checkpoint benchmark results are separate from the conversion/quantization measurements in this card; no identical quantized benchmark scores are implied. ModelScope is not published yet. QwenPaw integration remains in Draft PR #8463; use the pinned setup guide and its companion patched runtime.

License

Apache 2.0; see LICENSE. FlexDM layout attribution and its MIT license are retained in THIRD_PARTY_LICENSES.md. The reference implementation is available in the original model repository.