Instructions to use GGQuanta/Xenomi-nano with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use GGQuanta/Xenomi-nano with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf GGQuanta/Xenomi-nano:BF16 # Run inference directly in the terminal: llama cli -hf GGQuanta/Xenomi-nano:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf GGQuanta/Xenomi-nano:BF16 # Run inference directly in the terminal: llama cli -hf GGQuanta/Xenomi-nano:BF16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf GGQuanta/Xenomi-nano:BF16 # Run inference directly in the terminal: ./llama-cli -hf GGQuanta/Xenomi-nano:BF16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf GGQuanta/Xenomi-nano:BF16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf GGQuanta/Xenomi-nano:BF16
Use Docker
docker model run hf.co/GGQuanta/Xenomi-nano:BF16
- LM Studio
- Jan
- vLLM
How to use GGQuanta/Xenomi-nano with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "GGQuanta/Xenomi-nano" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "GGQuanta/Xenomi-nano", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/GGQuanta/Xenomi-nano:BF16
- Ollama
How to use GGQuanta/Xenomi-nano with Ollama:
ollama run hf.co/GGQuanta/Xenomi-nano:BF16
- Unsloth Studio
How to use GGQuanta/Xenomi-nano with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for GGQuanta/Xenomi-nano to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for GGQuanta/Xenomi-nano to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for GGQuanta/Xenomi-nano to start chatting
- Pi
How to use GGQuanta/Xenomi-nano with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf GGQuanta/Xenomi-nano:BF16
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "GGQuanta/Xenomi-nano:BF16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use GGQuanta/Xenomi-nano with Docker Model Runner:
docker model run hf.co/GGQuanta/Xenomi-nano:BF16
- Lemonade
How to use GGQuanta/Xenomi-nano with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull GGQuanta/Xenomi-nano:BF16
Run and chat with the model
lemonade run user.Xenomi-nano-BF16
List all available models
lemonade list
- Hermes Agent
How to use GGQuanta/Xenomi-nano with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf GGQuanta/Xenomi-nano:BF16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default GGQuanta/Xenomi-nano:BF16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use GGQuanta/Xenomi-nano with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf GGQuanta/Xenomi-nano:BF16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "GGQuanta/Xenomi-nano:BF16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
玄幂 Xenomi-nano(0.8B)GGUF
Xenomi-nano 是玄幂家族的端侧档,面向路由、术语抽取、关键词与短文本判别。
本仓库提供合并后的 BF16 GGUF,可用 llama.cpp / llama-cli / llama-server 直接加载。
- 出品: 北京中科国光量子科技有限公司(国光量子 / GGQUANTA)
- 基座:
Qwen/Qwen3.5-0.8B(Apache 2.0) - 训练: 垂类数据 + LoRA / SFT,再 merge 为完整权重并转为 GGUF
- 角色: 端侧 / 低延迟判别,不是通用大模型旗舰档
同家族:4B = Xenomi-mini,27B = Xenomi-pro(规划),122B-A10B = Xenomi-max(规划)。
文件
| 文件 | 格式 | 说明 |
|---|---|---|
xenomi-0.8b-p0-cuda-BF16.gguf |
GGUF BF16 | 完整合并权重;约 1.4 GB |
本文件不含量子路由头(方案 C 的读出头是独立模块,不在 GGUF 内)。
身份金句门(thin-A)是服务侧规则,推理权重本身不内置拦截。
重要声明
- 相对基座的准确率优势归因于垂类数据与 LoRA / SFT,不归因于量子硬件或「量子优越性」。
- 下列分数来自训练侧 CUDA LoRA(PEFT) 对照,冻结 eval v1,平台
wsl-rtx5070,日期 2026-07-28。本 GGUF 为对应 adapter 的合并转出,未在 llama.cpp 上重跑同一套 eval。 - Acc / F1 按任务分别呈现,不能加总为「通用能力分」。
评测(相对 Qwen3.5-0.8B,eval v1)
| 模型 | 路由 Acc | RAG Acc | NER micro-F1 | Keyword F1 | 短文本 Acc | Parse-OK |
|---|---|---|---|---|---|---|
| Qwen3.5-0.8B | 31.2 | 80.0 | 33.7 | 22.2 | 56.2 | 99.2 |
| Xenomi-nano | 95.0 | 83.8 | 74.7 | 74.1 | 100.0 | 100.0 |
相对基座:路由 +63.8、RAG +3.8、NER +41.0、关键词 +51.9、短文本 +43.8。五项门槛均为 PASS。
产品身份(叠加 thin-A v1.2):Identity **92%**。
用法(llama.cpp)
# 交互式 Chat
llama-cli -hf GGQuanta/Xenomi-nano:BF16 -cnv --jinja --reasoning off
# 或本地文件
llama-cli -m xenomi-0.8b-p0-cuda-BF16.gguf -cnv --jinja --reasoning off -c 8192
# OpenAI 兼容服务
llama-server -m xenomi-0.8b-p0-cuda-BF16.gguf --jinja --port 8080
Chat 模板为 Qwen3.5 / ChatML。默认可关思考(--reasoning off);需要 <think> 时设 --reasoning on。
建议系统提示(可选):
你是玄幂(Xenomi),由中科国光量超训练的量子计算 / 科研领域模型。
采样可从非思考档起步:temperature=0.7,top_p=0.8,top_k=20。
许可
基座与本仓库权重按 Apache 2.0 提供。使用时请同时遵守 Qwen3.5 的许可与使用政策。
- Downloads last month
- 42
16-bit