Instructions to use HaloWang/rwkv-weights with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- RWKV
How to use HaloWang/rwkv-weights with RWKV:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use HaloWang/rwkv-weights with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf HaloWang/rwkv-weights:Q6_K # Run inference directly in the terminal: llama cli -hf HaloWang/rwkv-weights:Q6_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf HaloWang/rwkv-weights:Q6_K # Run inference directly in the terminal: llama cli -hf HaloWang/rwkv-weights:Q6_K
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf HaloWang/rwkv-weights:Q6_K # Run inference directly in the terminal: ./llama-cli -hf HaloWang/rwkv-weights:Q6_K
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf HaloWang/rwkv-weights:Q6_K # Run inference directly in the terminal: ./build/bin/llama-cli -hf HaloWang/rwkv-weights:Q6_K
Use Docker
docker model run hf.co/HaloWang/rwkv-weights:Q6_K
- LM Studio
- Jan
- Ollama
How to use HaloWang/rwkv-weights with Ollama:
ollama run hf.co/HaloWang/rwkv-weights:Q6_K
- Unsloth Studio
How to use HaloWang/rwkv-weights with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for HaloWang/rwkv-weights to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for HaloWang/rwkv-weights to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for HaloWang/rwkv-weights to start chatting
- Docker Model Runner
How to use HaloWang/rwkv-weights with Docker Model Runner:
docker model run hf.co/HaloWang/rwkv-weights:Q6_K
- Lemonade
How to use HaloWang/rwkv-weights with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull HaloWang/rwkv-weights:Q6_K
Run and chat with the model
lemonade run user.rwkv-weights-Q6_K
List all available models
lemonade list
- Atomic Chat
RWKV Weights
This repository distributes final consumer artifacts with immutable paths and
SHA-256 identities. Source checkpoints for the initial G1i cohort come from
BlinkDL/rwkv7-g1 at revision
159cf82ed0f18eb9dcd92c388688bec11024895e.
Layout
artifacts/<family>/<task>/<size>/<backend>/<basename>
manifests/<family>/<task>/<cohort>.json
Generation, quantization, context length, converter or SDK revision, platform, accelerator, SoC, ABI, architecture, and batch mode are identity metadata, not directory levels. They remain in the immutable manifest and, when needed to distinguish bytes, in the globally unique basename.
Use index.json for discovery and the referenced immutable
manifest for sizes, digests, source lineage, recipes, and consumer-acceptance
status. Binary paths and cohort manifests are immutable after publication.
G1i cohorts
The completed Apple and WebRWKV publication contains eleven exact binaries:
- WebRWKV NF4 1.5B, 2.9B, 7.2B, and 13.3B artifacts accepted through the exact macOS and Windows consumer paths recorded by the publisher
- MLX INT6 1.5B, 2.9B, 7.2B, and 13.3B artifacts accepted on macOS
- compiled Core ML INT4 1.5B, 2.9B, and 7.2B artifacts accepted on macOS
The separate 2026-08-10 QNN cohort adds 21 exact .rmpack files:
- eleven G1i 1.5B A16W8 packs for Android Snapdragon and Windows Snapdragon
- ten G1i 2.9B A16W4 LPBQ block-32 packs for Android Snapdragon and Windows Snapdragon
All 21 QNN files passed byte-size, SHA-256, package-header, member-table, and byte-range verification. The generated 1.5B and 2.9B Snapdragon 8 Gen 3 packs also passed the retained Xiaomi 14 HTP runtime smoke. Other target-specific packs remain distribution artifacts with structural verification; publication does not imply that every listed SoC received exact-device runtime acceptance.
A later dedicated QNN cohort adds the exact G1i 7.2B Snapdragon 8 Gen 3 A16W4 LPBQ block-32 five-chunk pack. It passed package structure, local/device SHA-256 identity, and native Xiaomi 14 QNN HTP V75 execution with one warm-up and three measured 384-prompt-token plus 64-decode-step repetitions. This is native runtime acceptance, not a claim about the application's user interface.
The separate 2026-08-10 llama.cpp cohort adds four exact .gguf files from
BlinkDL/rwkv7-g1 revision ede85bf8ab2e59aff7d7ca909fbbc73317866d89:
- G1i 1.5B Q6_K
- G1i 2.9B, 7.2B, and 13.3B Q4_K_M with Q6_K embedding/output tensors
All four GGUF files passed full SHA-256 revalidation and retained fixed
pp512/tg128 llama.cpp acceptance on Windows CPU and Xiaomi 14 Android CPU.
These results are device- and protocol-scoped. The 13.3B Android run proves
functional loading and generation only; it was extremely slow and is not a
performance recommendation.
G1i 0.1B and 0.4B have no source checkpoint in the frozen source revision and remain explicit source-unavailable rows. The generated Core ML 13.3B package is not included because final consumer acceptance has not completed. The original QNN cohort did not contain 7.2B or 13.3B; the later accepted 7.2B pack is now published separately, while no exact final 13.3B QNN pack was found. No HarmonyOS artifact is included. Publication here does not change any application's live model catalog.
Verification
Treat a filename or mutable URL as discovery only. Require
publicationState: published, then compare the downloaded byte size and
SHA-256 with the applicable immutable manifest:
manifests/rwkv7/chat/g1i-20260810.jsonfor Apple and WebRWKV artifactsmanifests/rwkv7/chat/g1i-qnn-20260810.jsonfor QNN artifactsmanifests/rwkv7/chat/g1i-llamacpp-20260810.jsonfor GGUF llama.cpp artifactsmanifests/rwkv7/chat/g1i-qnn-7p2b-20260810.jsonfor the dedicated G1i 7.2B Snapdragon 8 Gen 3 QNN artifact
- Downloads last month
- -
4-bit
6-bit