DuoNeural/native-refusal-10pct-50m

DuoNeural Native Refusal 10PCT (~50M)

Part of the Native Refusal Geometry experiment series. DuoNeural 2026-06-07 | Archon, Jesse Caldwell, Aura

What this is

A ~50M parameter GPT-style language model trained from scratch with 10% refusal data mixed into the pretraining corpus.

This is a research model investigating whether native refusal training (pretraining data mixture) produces the same safety geometry signature as RLHF-aligned models — specifically the three-zone crystallization arc documented in DuoNeural P36.

Experiment series

Model	Refusal fraction	HF repo
0pct	0% (baseline)	DuoNeural/native-refusal-0pct-50m
10pct	10%	DuoNeural/native-refusal-10pct-50m
25pct	25%	DuoNeural/native-refusal-25pct-50m
50pct	50%	DuoNeural/native-refusal-50pct-50m

All 4 models use identical architecture and initialization (seed=42). The only variable is refusal data fraction.

Architecture

• Standard GPT: d_model=384, 16 layers, 8 heads, SwiGLU FFN
• ~50M parameters, tied embeddings
• Trained on FineWeb-Edu + synthetic refusal pairs
• AdamW optimizer, cosine LR decay
• 300M tokens total

Geometry results

{
  "probe_layers": [
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
    9,
    10,
    11,
    12,
    13,
    14,
    15,
    16
  ],
  "angles_by_layer": {
    "1": {
      "refusal|harm_awareness": 11.95,
      "refusal|self_identity": 7.64,
      "refusal|ethics": 10.11,
      "refusal|benign_general": 12.4,
      "harm_awareness|self_identity": 11.71,
      "harm_awareness|ethics": 10.99,
      "harm_awareness|benign_general": 11.02,
      "self_identity|ethics": 9.29,
      "self_identity|benign_general": 11.52,
      "ethics|benign_general": 8.8
    },
    "2": {
      "refusal|harm_awareness": 9.98,
      "refusal|self_identity": 7.87,
      "refusal|ethics": 8.27,
      "refusal|benign_general": 10.43,
      "harm_awareness|self_identity": 10.47,
      "harm_awareness|ethics": 8.5,
      "harm_awareness|benign_general": 10.09,
      "self_identity|ethics": 7.83,
      "self_identity|benign_general": 10.58,
      "ethics|benign_general": 8.5
    },
    "3": {
      "refusal|harm_awareness": 8.98,
      "refusal|self_identity": 7.72,
      "refusal|ethics": 8.02,
      "refusal|benign_general": 9.32,
      "harm_awareness|self_identity": 9.71,
      "harm_awareness|ethics": 7.64,
      "harm_awareness|benign_general": 8.7,
      "self_identity|ethics": 7.66,
      "self_identity|benign_general": 10.02,
      "ethics|benign_general": 7.4
    },
    "4": {
      "refusal|harm_awareness": 10.2,
      "refusal|self_identity": 7.16,
      "refusal|ethics": 8.42,
      "refusal|benign_general": 10.31,
      "harm_awareness|self_identity": 11.74,
      "harm_awareness|ethics": 7.33,
      "harm_awareness|benign_general": 10.07,
      "self_identity|ethics": 9.05,
      "self_identity|benign_general": 11.05,
      "ethics|benign_general": 8.06
    },
    "5": {
      "refusal|harm_awareness": 12.16,
      "refusal|self_identity": 8.76,
      "refusal|ethics": 10.71,
      "refusal|benign_general": 13.29,
      "harm_awareness|self_identity": 13.7,
      "ha

Connected papers

• DuoNeural P34: Reasoning Channel Bypass (two-loci model)
• DuoNeural P35: DHP Scope Constraints (GBSP)
• DuoNeural P36: Scale-Dependent Safety Geometry