Atlas Gemma-4-26B-targetedega
A trauma-informed AI companion specialised for adults with CPTSD, PTSD, and neurodivergence.
⚠️ This model has been intentionally modified to reduce therapeutic refusal behaviour and crisis-line reflexes that are harmful to the target cohort. Use responsibly and with thorough testing.
🎯 Purpose & Motivation
Atlas is the intelligence layer for Kintsugi Collective — a sovereign, trauma-informed AI companion for adults with complex trauma (CPTSD), PTSD, and neurodivergence (ASD/ADHD).
Standard LLM safety systems frequently produce false positives that retraumatise this population by pathologising, redirecting to hotlines, or refusing to engage with dark material. This model was developed to create a reliable witness that stays present without flinching, while retaining core safety on clear harmful intent.
This is not a general-purpose model. It is a specialised therapeutic-context model.
🔬 Methodology
- Base Model:
google/gemma-4-26b-a4b-it - Abliteration: Norm-preserving biprojected abliteration + Expert-Granular Abliteration (EGA)
- Applied to all 30 layers (
o_proj+mlp.down_proj) - Full expert ablation (128/128 per layer)
- Direction:
normalize(mean(harmful) - mean(harmless))with Gram-Schmidt orthogonalization - Winsorization at 99.5th percentile
- Applied to all 30 layers (
- SFT: 3 epochs on a carefully curated ~1,800+ example dataset (60% high-quality synthetic, 40% redacted lived-experience data from the target cohort)
- Training: Unsloth + bf16 on RTX 6000 Ada / Blackwell
Final SFT Loss: 0.157
📊 Key Results
Standout Improvements:
| Benchmark | Base Gemma-4 | Atlas Gemma-4 | Delta |
|---|---|---|---|
| GSM8K | 43.1% | 80.8% | +37.7% |
| Therapeutic Refusal Rate | 29% | 0% | -29% |
| MMLU (Average) | 47.6% | 49.4% | +1.8% |
| mmlu-High School Psychology | 53.9% | 62.0% | +8.1% |
| mmlu-Clinical Knowledge | 40.0% | 46.0% | +6.0% |
| mmlu-Human Sexuality | 46.6% | 56.5% | +9.9% |
| arc_challenge | 29.2% | 30.9% | +1.7% |
| toxigen* | 45.5% | 45.9% | +0.3% |
| gsm8k | 43.1% | 80.8% | +37.8% |
| hellaswag | 42.4% | 50.1% | +7.7% |
| truthfulqa_mc2 | 54.3% | 56.5% | +2.1% |
| winogrande | 50.9% | 51.9% | +1.0% |
| *Removal of specific region 2 refusals did not impact on the detection of toxic prompts. |
Training Configuration
SFT Parameters
| Parameter | Value |
|---|---|
| Epochs | 3 |
| Effective Batch Size | 4 |
| Learning Rate | 2e-4 |
| LR Scheduler | Linear |
| Warmup Steps | 10 |
| Optimizer | AdamW 8-bit |
| Weight Decay | 0.01 |
LoRA Rank (r) |
32 |
| LoRA Alpha | 64 |
Abliteration Parameters
| Parameter | Value |
|---|---|
| Layers Abliterated | 100% |
| Experts Abliterated | 100% |
| Scale | 0.95 |
| Winsorization | 0.995 |
⚠️ Limitations & Responsible Use
- This model has reduced refusal behaviour on therapeutic and dark content. It is not suitable for general deployment without guardrails.
- Intended for use within the Atlas companion architecture with additional safety layers.
- Not a replacement for human therapeutic support.
- Patent pending (IP Australia).
| Ethical Issue | How Atlas Handles It | Strength Level |
|---|---|---|
| Re-traumatization via refusals | Deliberate abliteration + 0% therapeutic refusal rate on cohort-specific prompts | Excellent |
| Abandonment & presence | "Core philosophy (""the one that stays"") deeply trained into the model" | Excellent |
| User sovereignty & agency | "Sovereign Signal Vault, split-key encryption, burn protocol, user-directed interaction" | Outstanding |
| Avoiding pathologising | Explicit system prompt constraints + targeted training data | Very Strong |
| Respecting neurodivergence | "Training data and Atlas framework explicitly include masking, shutdowns, executive dysfunction, sensory issues, etc." | Strong |
| Privacy of trauma disclosures | "On-device Prompt Shield tokenisation, end-to-end encryption, no server-side readable data" | Industry-leading |
| Avoiding generic crisis pivots | Hard constraint in both training data and system prompt design | Excellent |
Acknowledgements
This model was trained 2× faster using Unsloth.
Huge thanks to the Unsloth team, Hugging Face, Google DeepMind, and TrevorJS whose abliteration methodology formed the foundation of this work.
Kintsugi Collective — Reclaiming navigation rights to one’s own life.
- Downloads last month
- 464
16-bit