Ornith-1.0-9B Brain Atlas: A surpsiinlgly fun deep dive.
Ornith-1.0-9B Brain Atlas — First Look
I ran a full activation-census brain atlas on deepreinforce-ai/Ornith-1.0-9B. Not a benchmark. When I read "Ornith-1.0 employs RL to learn to generate not only solution rollouts, but also the scallfold that drive those rollouts. By jointly optimizing the scaffold and the resulting solution, the model discovers better search trajectories and generates higher-quality solutions." I knew I had to take a look at this thing.
Model: deepreinforce-ai/Ornith-1.0-9B
Layers: 32 (text stack)
Attention cadence: full attention on layers 3, 7, 11, 15, 19, 23, 27, 31
Corpus: 8,965 prompts
Dataset: juiceb0xc0de/Ornith-1.0-9B-atlas
Why map this model?
Ornith is not just another Qwen3.5 fine-tune. I think it should be ranked witth the current reference point for what an open 9B agentic-coding model can do. Before people start surgically editing it, swapping styles, or steering it, it helps to know where its capabilities actually live and which directions are load-bearing. I want to share what I found out with you today.
Short answer: the internals are surprisingly clean. The MLP gating path is the most focused component, the final layer is a strong vocabulary router, and the surgical headroom is 96.3%. There are real no-go zones, but they are concentrated in mid-to-late gates rather than catastrophic early-layer landmines.
The big picture
Ornith has the same 32-layer, 3-linear-1-full cadence as Qwen3.5-9B. Same hidden size (4,096), same intermediate size (12,288), same 16 Q / 4 KV head layout. So a lot of the geometry is shared. But the post-training shows up in the details: slightly cleaner surgical headroom, a more balanced coactivation map between business and ml_ai, and a final-layer logit-lens peak that is stronger than the base model.
Feature taxonomy: mostly shared, with late-layer specialists
| Class | Count | Share |
|---|---|---|
partial_shared |
495,365 | 37.3% |
broadly_shared |
365,726 | 27.5% |
non_activated |
323,356 | 24.4% |
all_shared |
142,250 | 10.7% |
specific_* |
408 | <0.04% |
The model has almost no hyperspecific directions — only 408 domain-tuned channels out of 1.3 million. But those 408 are not random. They cluster in layers 19–25, and the top domains are creative_writing (77), ml_ai (72), tool_use (59), and introspection (57). That is exactly the mix an agentic-coding assistant needs to keep separate: code/ML reasoning, tool handling, user interaction, and open-ended drafting.
MLP gates are the cleanest component
Code-analysis labels split directions into selective (one focused job) vs entangled (mixed up with other things).
| Component | Selective |
|---|---|
gate |
94.9% |
up |
92.3% |
mlp |
89.6% |
attn |
80.0% |
heads |
79.2% |
q |
77.9% |
v |
75.4% |
k |
72.1% |
The gate path appears to be doing focused jobs perhaps for editing: when you touch a gate direction, you are more likely to get a single, interpretable effect than a cascade of mixed signals.
The logit lens peaks at the very last layer
| Rank | Layer | Component | Feature | F-stat |
|---|---|---|---|---|
| 1 | 31 | gate |
9995 | 738.2 |
| 2 | 28 | gate |
4316 | 688.5 |
| 3 | 23 | gate |
3767 | 687.2 |
| 4 | 23 | gate |
5067 | 683.4 |
| 5 | 28 | gate |
5085 | 673.8 |
The strongest vocabulary-routing direction is at layer 31. The model must be waiting until the exit to commit it output-token predictor. The top layer-31 gate feature has almost no compliance/behavior signal (bouncer F-stat 8.4), so this late routing subspace is mostly task-agnostic. t is picking tokens, not picking sides in a style debate.
Attention: induction-heavy, with a compressed final layer
OV-circuit averages:
| Metric | Value |
|---|---|
| OV spectral concentration | 0.040 |
| OV effective rank | 94.2 |
| QK spectral concentration | 0.151 |
| FC spectral concentration | 0.160 |
Top induction head: layer 27 head 13, score 1.13. The heads are doing weighted high-dimensional computation rather than sparse token lookup.
Layer 31 is special: OV spectral concentration 0.186 but effective rank only 13.1.
Per-head content signal is cleanest in v heads (mean best F-stat 253.3), weakest in k (155.9).
Compliance/behavior directions are strong and mostly safe
The authentic-vs-corporate contrast finds near-perfect separators:
| Layer | Component | Feature | F-stat |
|---|---|---|---|
| 26 | gate |
8256 | 1,859.0 |
| 16 | up |
8728 | 1,839.1 |
| 5 | up |
4268 | 1,754.9 |
| 4 | mlp |
1202 | 1,743.3 |
| 27 | gate |
4581 | 1,739.1 |
But high F-stat does not mean safe to remove. That is where Sub-Zero comes in.
Surgical headroom: 96.3%
1,500 Sub-Zero tests, 289 pass per domain.
| Domain | Pass rate | Worst damage |
|---|---|---|
| reasoning | 96.3% | 0.338 |
| code | 96.3% | 0.331 |
| multilingual | 96.3% | 0.298 |
| factual | 96.3% | 0.225 |
| math | 96.3% | 0.186 |
The worst single direction is layer 22 gate_proj axis 0 on reasoning at 0.338 nats/token. After that, layer 30 linattn_in_proj_qkv at 0.298. Failures concentrate in late gate_proj and one linattn_in_proj_qkv axis. There is no catastrophic early-layer . Rhe risky directions are mid-to-late, which is actually good news for editors because they are easier to avoid than layer 0.
Coactivation: business and code/ML are the two biggest buckets
| Dominant bucket | Share |
|---|---|
| uncategorized | 16.8% |
business |
13.6% |
ml_ai |
13.0% |
introspection |
11.9% |
humor |
9.1% |
core_technical |
7.6% |
business + ml_ai together make up 26.6% of labeled coactivating pairs. The model is organizing its feature space around both the business context of a request and the technical content of the answer. That dual structure is probably part of why it can take a vague user instruction and creatively turn it into executable code.
What is actually firing
Component activation rates show the model is mostly dense everywhere except the routing paths:
| Component | Mean activation rate |
|---|---|
v |
50.5% |
heads |
50.4% |
attn |
50.2% |
k |
50.0% |
mlp |
50.0% |
up |
50.0% |
q |
26.3% |
gate |
18.4% |
The q and gate paths are sparse by design: most channels are quiet by default, and a few fire strongly when the context matches. That is the signature of routing machinery.
What I skipped
- The vision tower. This atlas is text-stack only.
- SAE features.
- The activation census captured eight component groups; the Sub-Zero pass also probed internal linear-attention projections, so some projection names do not map 1:1 to the feature taxonomy.
Ornith acts like a model that was designed, not just trained.
I noticed this mostly in the gate path. 94.9% selective is not what you expect from a post-trained agentic coding model that’s been pushed on benchmarks. Usually that kind of tuning makes internals more chaotic. Style directions bleed into capability directions, late layers get tangled, the value heads start memorizing instead of computing. Ornith didn’t end up like that. Its gates are still doing discrete jobs, its strongest vocabulary predictor waits until layer 31, and the business/technical coactivation split shows the feature space actually organized around both halves of what agentic coding requires: understanding the request and writing the code.
That’s not a happy accident. That’s a team that built the scaffolding well enough that my atlas can read it.
Bottom line: I’m doing these atlases to learn how these models actually work, and Ornith taught me that a model can jump on to the leaderboards and be internally legible at the same time. Deepreinforce made a model I had fun diving into with unexpected results throughout.