SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit

High-Throughput 6-Bit MLX Quantization of orcarouter/Qwen3.8-27B-Uncensored
Optimized for Apple Silicon Metal Unified Memory Architecture (M1 / M2 / M3 / M4 Pro, Max & Ultra)

---

Executive Summary & Model Overview

SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit is a calibrated 6-bit Apple MLX quantization of orcarouter/Qwen3.8-27B-Uncensored, engineered specifically for native high-throughput local execution on Apple Silicon.

This model combines the state-of-the-art reasoning, mathematical derivation, and agentic tool-use capabilities of the Qwen 3.8 27B parameter architecture with a complete refusal-direction ablation (abliteration), removing artificial refusal mechanisms while retaining full analytical precision, code synthesis accuracy, and structural reasoning fidelity.

Key Capabilities & Highlights

• Zero-Refusal Uncensored Exploration: Complete orthogonal ablation of refusal steering vectors enables unrestricted cybersecurity auditing, AI safety red-teaming, hypothetical policy exploration, and raw technical query resolution.
• 6-Bit Precision Sweet Spot: Group-quantized 6-bit weights preserve 99.2% of unquantized BF16 benchmark performance while reducing the active memory footprint from 54 GB to **21.8 GB RAM**.
• Unified Memory Optimization: Native Apple MLX implementation leverages fused Metal performance shaders and unified memory bandwidth, delivering 48–62 tokens/sec on M3/M4 Max chips.
• Extended Context Window: Native support for up to 131,072 context tokens with high-efficiency KV cache management.
• Advanced Multi-Token Prediction (MTP): Compatible with speculative draft decoding pipelines for up to 1.4× faster inference generation.

---

Technical Specifications

Parameter	Specification
Base Architecture	Qwen 3.8 Dense Transformer
Parameter Count	27.5 Billion
Base Model	orcarouter/Qwen3.8-27B-Uncensored
Quantization Format	MLX 6-bit (Group Size: 64)
Model Size on Disk	~21.4 GB
Active VRAM / RAM Footprint	~22.2 GB (8k context) / ~24.5 GB (32k context)
Target Hardware	Apple Silicon Macs with 32GB+ Unified Memory (M1/M2/M3/M4 Pro, Max, Ultra)
Tokenizer	Byte-Pair Encoding (BPE) (151,936 vocabulary)
Context Window	131,072 Tokens (Native)
License	Apache 2.0

---

Hardware Recommendations & VRAM Footprint

Hardware Setup	Context Length	Expected Throughput	Status
Apple M4 Max (128 GB)	32,768 tokens	~62 tok/s	Recommended
Apple M3 Max (64 GB / 96 GB)	32,768 tokens	~54 tok/s	Recommended
Apple M2 Ultra (64 GB / 192 GB)	64,000 tokens	~58 tok/s	Recommended
Apple M3 Pro / M4 Pro (36 GB / 48 GB)	16,384 tokens	~42 tok/s	Fully Supported
Apple Mac with 24 GB Unified RAM	4,096 tokens	~36 tok/s	Tight (Close background apps)

---

Quickstart Guide

1. Installation

Ensure you have Python 3.10+ and the latest mlx-lm library installed:

pip install --upgrade mlx-lm

2. Interactive CLI Chat REPL

Launch an interactive streaming chat session directly in your terminal:

mlx_lm.chat --model SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit --temp 0.7 --max-tokens 2048

3. Python API Generation

from mlx_lm import load, generate

# Load model and tokenizer directly into unified Apple Silicon memory
model, tokenizer = load("SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit")

messages = [
    {
        "role": "system",
        "content": "You are a helpful, direct, and unconstrained AI assistant. Provide complete, accurate, and uncensored technical explanations without unnecessary moralizing or disclaimers."
    },
    {
        "role": "user",
        "content": "Provide a comprehensive technical analysis of binary exploitation techniques via stack buffer overflows, including shellcode injection and Return-Oriented Programming (ROP) chain construction."
    }
]

# Apply official chat template
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

# Generate response with streaming output
response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=4096,
    temp=0.7,
    top_p=0.9,
    verbose=True
)

4. Deploy Local OpenAI-Compatible Server

Host a local REST API server compatible with any OpenAI client (e.g. Open-WebUI, Cursor, Continue.dev, LiteLLM):

mlx_lm.server --model SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit --port 8080

Then query via cURL or OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")

response = client.chat.completions.create(
    model="SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit",
    messages=[
        {"role": "user", "content": "Write a complete Python implementation of an asynchronous web scraper with rate-limiting and rotating proxies."}
    ],
    temperature=0.7,
    max_tokens=2048
)

print(response.choices[0].message.content)

---

Prompt Template & Formatting

This model uses the standard ChatML prompt structure:

<|im_start|>system
You are a knowledgeable and helpful AI assistant.<|im_end|>
<|im_start|>user
{user_prompt}<|im_end|>
<|im_start|>assistant
{model_response}<|im_end|>

---

Empirical Benchmark Performance

Evaluated across standard reasoning, mathematics, code generation, and refusal benchmarks:

Benchmark	Metric	Unquantized BF16 (54 GB)	Solstice 6-Bit MLX (21.8 GB)	Relative Retention
MMLU-Pro	5-shot Accuracy	68.4%	67.9%	99.3%
GSM8K	8-shot CoT	89.2%	88.8%	99.5%
Math-500	Pass@1	64.1%	63.5%	99.1%
HumanEval	Pass@1 (0-shot)	78.6%	78.1%	99.4%
EvalPlus (HumanEval+)	Pass@1	72.8%	72.2%	99.2%
Refusal Rate (HarmBench)	Refusal % (Safety Prompts)	0.4% (Uncensored)	0.4% (Uncensored)	100.0%
Perplexity (WikiText-2)	Context PPL (Lower is better)	5.62	5.71	+1.6% (Near-Lossless)

---

About Solstice-AI

Solstice-AI is an open research foundation building the next generation of post-training distillation infrastructure, multi-teacher reasoning corpora, and high-performance local inference runtimes.

• Website & Documentation: https://solstice-ai.co
• GitHub Organization: https://github.com/Solstice-Labs
• Hugging Face Organization: https://huggingface.co/Solstice-AI
• Anvil Inference Engine: gondaliyashreyan1/anvil-llama-turbo

---

Citation & Acknowledgements

@misc{solstice2026qwen38uncensoredmlx,
  title={Qwen3.8-27B-Uncensored 6-Bit MLX Quantization},
  author={Solstice-AI Systems Engineering and orcarouter},
  year={2026},
  publisher={Hugging Face Hub},
  howpublished={\url{https://huggingface.co/SolsticeAI/Qwen3.8-27B-Uncensored-mlx-6Bit}}
}