Instructions to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6", filename="BugTraceAI-CORE-Ultra-SFT-Q6_K.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K # Run inference directly in the terminal: llama cli -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K # Run inference directly in the terminal: llama cli -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K # Run inference directly in the terminal: ./llama-cli -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K # Run inference directly in the terminal: ./build/bin/llama-cli -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Use Docker
docker model run hf.co/BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
- LM Studio
- Jan
- Ollama
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with Ollama:
ollama run hf.co/BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
- Unsloth Studio
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 to start chatting
- Pi
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Run Hermes
hermes
- Atomic Chat new
- OpenClaw new
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with Docker Model Runner:
docker model run hf.co/BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
- Lemonade
How to use BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull BugTraceAI/BugTraceAI-CORE-Ultra-27B-Q6:Q6_K
Run and chat with the model
lemonade run user.BugTraceAI-CORE-Ultra-27B-Q6-Q6_K
List all available models
lemonade list
This model makes no sense
Is this a finetuned model? If so why is it catagorized as a quantization of (DavidAU/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking).
Did you just rename their model after quantizing it and not change anything? Because all that does is confuse people as to the purpose of your model.
Please clarify this issue
Hey rombodawg β fair to raise the question, so let me be transparent about what actually happened here.
This is a fine-tune, not a rename. The "quantization of" tag in the HuggingFace metadata is auto-generated based on the base_model field β it doesn't capture the full lineage. Here's the actual pipeline:
Base selection β We chose DavidAU/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking specifically because the community was asking for Qwen3 architecture (see discussion #1). Starting from an already-abliterated base means we're not wasting compute re-teaching it to be uncensored.
Custom dataset β 2,541 training examples curated from: HackerOne disclosed reports, CVE writeups, Bug Bounty Daily CoT chains, GitHub security research (2024β2026). Not publicly available as a single dataset anywhere. This took weeks of ETL pipelines, scraping, normalization, and DPO-format conversion (including a β tag migration for Qwen3's native vocab).
Training β SFT via Unsloth on a RunPod H100 80GB, LoRA r=16 injected across all 7 attention + MLP modules across 64 transformer layers, 2 epochs. The VRAM math alone required significant engineering work (54GB base model + optimizer states).
Merge + IMatrix Quant β LoRA merged back to BF16 full weights, then quantized with IMatrix guidance via llama.cpp to Q4_K_S and Q6_K.
Benchmarked β 5/5 on our BugTraceAI Ultra Bench v1.0 (Nuclei templates, CVE PoCs, JWT cracking, kernel exploits) at 0% refusal rate.
All of this is already documented in the model card under π§ Training Details β worth a read before assuming. The base model is the foundation, not the product. The work is in the data, the training objective, and what the model now does that the base didn't.
Can we possibly have the q8 quantized version as well? q6 seems to be hallucinating and jumping to conclusions. It falsely identified cloakbrowser as fully open source when it's only half open source and the actual browser build is closed source proprietary . Overall it seems weaker than the q8 qwen 27B I've been using. It degraded almost to ornith 35B MoE level where it had only shallow reasoning and didnt look deep into things
(Btw I'm sure nobody wants to hear this, but as far as "consumer grade GPUs" are concerned, an a6000 with 48GB of VRAM now counts as high end consumer GPU, a 4090 with 24GB of VRAM is considered entry level, and anything below that... well.... it's not pretty. What is a single rtx 6000 pro considered? Something like a gaming rig or stuff for creators to use. Now, only MULTIPLE rtx 6000 pros are considered "out there". I say this because I learned the hard way that the minimum quality a gguf can have is q8 to retain some semblance of the awesomeness of the full weights... )