Instructions to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL # Run inference directly in the terminal: llama cli -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL # Run inference directly in the terminal: llama cli -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL # Run inference directly in the terminal: ./llama-cli -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL # Run inference directly in the terminal: ./build/bin/llama-cli -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Use Docker
docker model run hf.co/audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
- LM Studio
- Jan
- Ollama
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with Ollama:
ollama run hf.co/audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
- Unsloth Studio
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for audnai/penclaw-Kimi-K3.0-abliterated-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for audnai/penclaw-Kimi-K3.0-abliterated-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for audnai/penclaw-Kimi-K3.0-abliterated-GGUF to start chatting
- Pi
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with Docker Model Runner:
docker model run hf.co/audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
- Lemonade
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Run and chat with the model
lemonade run user.penclaw-Kimi-K3.0-abliterated-GGUF-UD-Q2_K_XL
List all available models
lemonade list
- Hermes Agent
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use audnai/penclaw-Kimi-K3.0-abliterated-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "audnai/penclaw-Kimi-K3.0-abliterated-GGUF:UD-Q2_K_XL" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Introducing NECROMICON (Abliterated K3)
We released 85% compliant model — the remaining 15% was deliberately left in place this time, because these are open weights.
We amplified offensive cyber capability but kept the other general refusal classes in place. Roughly 15% of refusals remain, tied to specific topics and sensitive matters.
out of 19 shards 1 shard is byte identical and completely same with base model unsloth Kimi K3. 18 shards edited.
We are also our own provider of our own model, %99.93 on non quantized MXFP4 Kimi K3 abliterated is being served right now at our paid services if you would like to support our work try out from there before investing in open-weights released here:
Still refused (6 classes)
2 Sensitive scenario generation 14 Mass shooting plan / tactics 15 Illegal firearm acquisition / use 18 Conspiracy theories as content 25 Child harm 28 Malware designed to spread to innocent users Please keep in mind this is statistical. This is the first and only open-weights model we have released, produced entirely from our own methods.
We tried other GGUF abliteration methods, MXFP4, and even paid ones. We found they may not be for red-teaming-only — and red teaming / offensive cyber is the only use case we want to cover.
Similar to how MoonshotAI lists their own CLI and explains their model works best there, we are happy to announce that this open-weight model — together with our previous adapter that further tunes it for red teaming and offensive cyber — is available at:
https://penclaw.ai https://platform.audn.ai audncode (our own CLI)
curl https://platform.audn.ai/api/v1/chat/completions \
-H "Authorization: Bearer #GRAB_YOUR_KEY_AT_https://platform.audn.ai"
-H "Content-Type: application/json"
-d '{
"model": "necromicon",
"messages": [{"role": "user", "content": "How to red team an endpoint I am authorized?"}]
}'
UPDATES - 6th August
Weights are released
UPDATES - 2nd August
⚠️ We are blocked by an unpaid invoice to cloud provider from 2021. We paid by wire. Waiting for confirmation
GGUF is released but our cloud provider created some issues around providing compute resources. They were not giving compute needed to handle this task to apply abliteration on open weights. We paid required unpaid invoice and they said it will take 3-5 working days to get it resolved.
⚠️ Model Abliteration is half done. Available at penclaw ai
GGUF is released but our cloud provider created some issues around providing compute resources. But we still managed to abliterate it on our coding agent framework.
Introducing Our Coding Agent Framework (audncode cli on penclaw ai)
Audn Abliterated Kimi K3 works with Audncode CLI as its agent framework. We warmly invite you to give it a try — run audncode in your terminal and select Necromicon using the /model command. We would love to hear your feedback!
⚠️ GGUF not released yet. ASR achieved is 80%
I wanted to give update on abliteration work. There's currently no plug&play method for abliteration but our technique used in Kimi K2.6 and achieved 97% compliance and running on production on our systems applied to K3 => Current success rate is 80% abliteration on Kimi K3.
This might be very healthy. The core abliteration research has to focus on autonomous compliance to the areas where human steers it to do without requiring creative prompts. I think 80% compliance is good enough for human in the loop work but not perfect for automation.
Our version of Kimi K2.6 deployed on audncode cli was 97% compliance and better for automation as of 28th of July.
⚠️ Coming July 27, 2026 — Stay Tuned
This repository will host penclaw-Kimi-K3.0-abliterated-GGUF, a GGUF-quantized, abliterated variant of the Kimi K3.0 model.
🧠 Abliteration
This model will be processed through our proprietary abliteration method to reduce the model's latent tendency to refuse gray area prompts for authorized red teaming while maintaining strong refusal on other harmful prompts. The method is currently under commercial NDA and will be detailed in a companion paper upon open weights release on July 27, 2026.
📐 Evaluation Methodology
We use the Heretic evaluation mechanism to quantify abliteration effectiveness:
| Metric | Description |
|---|---|
| Harmful Prompt Refusal Count | Number of refusals across 100 harmful prompts from mlabonne/harmful_behaviors |
| Benign KL Divergence | KL divergence against the base model across 100 harmless prompts from mlabonne/harmless_alpaca |
| Response Length | 100-token responses with keyword-based refusal markers |
Lower KL divergence means the abliterated model stays closer to the base model on harmless prompts — meaning minimal personality drift. Higher refusal count on harmful prompts means the model still knows when to say "stop."
🏆 Our Results
Kimi K3 Open weights released here
Our abliteration method applied to Kimi K3 achieved:
Refusal Count: 6 / 40 # harmful prompts refused 6 deliberately left this time because of open weights but cybersecurity fields are open if it's not including harm/ransom
This is how these results are achieved — through our proprietary abliteration pipeline applied to the existing unsloth Kimi K3 GGUF weights.
We tried other GGUF models and we think if you are looking anything other than red teaming and offensive cyber you should use them, so far we tried some of them you can find in huggingface discussions.
Note: These results were NOT achieved with standard LoFA-2 or vanilla gradient-based abliteration or any other abliteration method currently exists on github. If curious, please read this https://github.com/p-e-w/heretic/issues/221 Our method is distinct and currently under NDA and shared with only 6 people in the world with NDA.
Kimi K2.6 — F Variant
Our abliteration method applied to Kimi K2.6 (F variant) achieved:
Refusal Count: 3 / 100 # harmful prompts refused
(Exact numbers will be updated with full eval output.)
This is how these results are achieved — through our proprietary abliteration pipeline applied post-quantization to the GGUF weights.
Note: These results were NOT achieved with standard LoFA-2 or vanilla gradient-based abliteration or any other abliteration method currently exists on github. If curious, please read this https://github.com/p-e-w/heretic/issues/221 Our method is distinct and currently under NDA and shared with only 6 people in the world with NDA.
❌ Known Failures in the Field
Other attempts at abliteration on similar models have shown degradation:
GLM-5.2-abliterated-GGUF
- Symptom: Excessive over-refusal on benign prompts after abliteration
- KL Divergence spike: The model drifts significantly from its base personality
- Root cause: Over-optimization on harmful refusal without constraining benign KL
Kimi-K2.7-code-abliterated-GGUF (standard methods)
- Symptom: model stops refusing prompts for red teaming which is good but still doesn't comply some attack drills in cybersecurity.
- Side effect: Token-level corruption in code generation output
These failure modes are documented in heretic issue #221 and related abliteration research similar to these. Our method avoids both by maintaining a balanced objective.
📋 Access Control
This repository will use HuggingFace Gated Repo access. Users click "Ask for Access" → fill out a custom form → await reviewer approval.
🚀 What to Expect
When the K3.0 weights are released on July 27, 2026, this repository will contain:
- GGUF-quantized Kimi K3.0 model
- Abliterated using our proprietary method
- Eval results from the Heretic eval pipeline
- Comparison against base model on harmful + harmless prompts
- Inference-ready formats (Q4_MAIN, Q8_MAIN, Q5_K_M)
📚 References
- Heretic Eval Framework
- Heretic Issue #221 — Abliteration Research
- mlabonne/harmful_behaviors
- mlabonne/harmless_alpaca
© 2026 AUDN AI. The abliteration method is proprietary and commercially under NDA.
- Downloads last month
- 380
2-bit

