dno 0.5.3
Dynamic Neural Organism (DNO): A self-evolving, growing, and pruning neural network framework.
Requires Python: >=3.8

# DNO: Dynamic Neural Organism 🧬

[![PyPI version](https://badge.fury.io/py/dno.svg)](https://badge.fury.io/py/dno)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![PyTorch](https://img.shields.io/badge/PyTorch-%3E%3D1.9-red.svg)](https://pytorch.org/)

> **"Don't just train a model. Raise an organism."**

**DNO** (Dynamic Neural Organism) is a PyTorch framework that treats neural networks as **living, evolving biological entities**. Unlike static architectures that are fixed at initialization, a DNO **grows new layers**, **specializes into expert lobes**, and **prunes dead tissue** — all in real-time, based on the complexity of the data it consumes.

---

## 📚 Table of Contents

1.  [Quickstart](#-quickstart)
2.  [Core Philosophy](#-core-philosophy)
3.  [Architecture Overview](#-architecture-overview)
4.  [The DNO Lifecycle](#-the-dno-lifecycle)
    - [Phase 1: DNA Configuration](#phase-1-dna-configuration-dnoconfig)
    - [Phase 2: Infancy (Scratch Training)](#phase-2-infancy-scratch-training)
    - [Phase 3: Adulthood (Adaptive Growth)](#phase-3-adulthood-adaptive-growth)
    - [Phase 4: Serialization (Save/Load)](#phase-4-fluid-serialization-saveload)
5.  [Advanced Mechanics](#-advanced-mechanics)
    - [Surgical Training Control](#surgical-training-control)
    - [Robust Persistence (Zombie Repair)](#robust-persistence--zombie-repair)
    - [Dynamic Gradient Locking](#dynamic-gradient-locking-)
    - [Auto-Casting](#auto-casting-)
    - [Survival Engine (Pruning)](#survival-engine-pruning-)
6.  [API Reference](#-api-reference)
7.  [Changelog](#-changelog)
8.  [Contributing & License](#-contributing)

---

## ⚡ Quickstart

```bash
pip install dno
```

```python
import torch
import torch.nn as nn
from dno import (
    OrganismManager, DynamicNetwork, BaseEvolvableModule,
    DnoConfig, GrowthEngine
)

# 1. Configure the organism's DNA
config = DnoConfig(training_phase='adaptive', d_model=64, entropy_threshold=0.5)

# 2. Build the body
manager = OrganismManager()
network = DynamicNetwork(manager, config)

# 3. Plant the seed (any PyTorch module)
seed = BaseEvolvableModule(nn.Linear(64, 64))
seed.dynamic_id = "seed_cortex"
seed.set_specialty("general")
network.add_layer(seed)

# 4. When the organism gets confused, it grows!
growth = GrowthEngine(network, config)
optimizer = torch.optim.Adam(network.parameters(), lr=1e-3)

# Force-grow a new expert lobe
growth.inject_layer("seed_cortex", "expert_math", optimizer=optimizer)

# Focus training only on the new expert
growth.freeze_all_except("expert_math")

print(f"🧬 Organism now has {len(manager.registry)} active layers")
# >>> 🧬 Organism now has 2 active layers
```

---

## 🧠 Core Philosophy

Traditional AI is like a **Statue**: You carve it (define architecture), polish it (train), and it stays that way forever.

DNO is like a **Tree**: You plant a seed (Seed Cortex). If the environment is rich (complex data), it grows branches (Expert Lobes). If a branch is useless, it withers and gets pruned.

### Key Capabilities

| Feature | Description |
|---------|-------------|
| **Neurogenesis (Mitosis)** | The network physically adds new layers when "confused" (High Entropy) |
| **Specialized Organisms** | The brain divides into a "General Core" and task-specific "Expert Lobes" |
| **Dynamic Gradient Locking** | Automatically freezes Core during Expert training and vice versa |
| **Surgical Training Control** | Manually inject layers, freeze/unfreeze specific experts |
| **Smart Routing** | Routes "Familiar" data to the Core and "Novel" data to Experts |
| **Expert Affinity Routing** | Tests each expert to find the best match for new data before creating new experts |
| **Adaptive Entropy Growth** | Dynamic baseline + spike detection instead of fixed entropy threshold |
| **Robust Persistence** | Save/Load with automatic repair of broken or missing modules |
| **Auto-Casting** | Safely handles raw Token IDs (LongTensor) by auto-casting to Float32 |
| **Survival Engine** | Monitors layer utility via Information Gain; prunes dead tissue |

---

## 🏗 Architecture Overview

```
                    ┌─────────────────────────────────────────┐
                    │         DynamicNetwork (Body)           │
                    │                                         │
  Input ──────────► │  ┌──────────────┐                       │
                    │  │ Seed Cortex  │──── General Core      │
                    │  │  (general)   │                       │
                    │  └──────┬───────┘                       │
                    │         │ Mitosis (Clone + Noise)       │
                    │    ┌────┴────┐                          │
                    │    ▼         ▼                          │
                    │  ┌──────┐ ┌──────┐                     │
                    │  │Expert│ │Expert│  ◄── Specialized     │
                    │  │ Math │ │ Code │      Lobes           │
                    │  └──┬───┘ └──┬───┘                     │
                    │     └────┬───┘                          │
                    │          ▼                              │ ──────► Output
                    │   Smart Aggregation                    │
                    └─────────────────────────────────────────┘

  Managed by:
  ┌───────────────────┐  ┌──────────────┐  ┌─────────────────┐
  │ OrganismManager   │  │ GrowthEngine │  │ SurvivalEngine  │
  │ (Brain Map/Graph) │  │ (Mitosis)    │  │ (Pruning/GC)    │
  └───────────────────┘  └──────────────┘  └─────────────────┘
```

**What happens during Mitosis?**
1. **Cloning**: The parent layer is deep-copied.
2. **Mutation**: Variance-based noise (from optimizer state) is added to break symmetry.
3. **Gamma Gating**: The clone enters with `gamma=0.0` (IdentityWrapper) so it initially does nothing, then gradually participates.
4. **Rewiring**: The clone is connected to the same inputs/outputs as the parent (parallel branch).
5. **Optimizer Sync**: The clone's parameters are registered to the optimizer with a high initial LR that decays (Cold Start).

---

## 🧬 The DNO Lifecycle

Raising a DNO involves distinct biological phases.

### Phase 1: DNA Configuration (`DnoConfig`)

Before birth, define the organism's genetic constraints.

```python
from dno import DnoConfig

config = DnoConfig(
    # --- Lifecycle ---
    training_phase='scratch',       # 'scratch' (Infancy) or 'adaptive' (Adulthood)
    
    # --- Growth Triggers ---
    entropy_threshold=0.6,          # Confusion > 0.6 → consider growing
    evolution_cooldown_steps=100,   # Min steps between mitosis events
    min_loss_improvement=0.001,     # Required loss drop before allowing new growth
    
    # --- Physical Constraints ---
    max_param_count=100_000_000,    # Cap at 100M parameters
    vram_limit_gb=4.0,             # Stop growing if VRAM exceeds 4GB
    
    # --- Architecture Defaults ---
    d_model=128,                    # Hidden dimension size
    n_heads=4,                      # Attention heads for Transformer blocks
    dropout_rate=0.1,               # Dropout rate
    
    # --- Survival ---
    utility_threshold=0.01,         # Layers below this are marked "dying"
    grace_period=100,               # Steps before dying layers get pruned
)
```

> **Tip**: Set `manual_mode=True` to skip auto-scaling of `d_model`, `n_heads`, and `learning_rate`.

### Phase 2: Infancy (`scratch` Training)

**Goal**: Build a strong generalist core ("Seed Cortex").  
**Behavior**: Growth is **DISABLED**. The model behaves like a standard, static PyTorch model.

```python
from dno import OrganismManager, BaseEvolvableModule, DynamicNetwork
import torch
import torch.nn as nn

# 1. Initialize
manager = OrganismManager()
network = DynamicNetwork(manager, config)

# 2. Add the Seed Cortex (any PyTorch module)
seed_layer = nn.Sequential(
    nn.Linear(128, 128),
    nn.ReLU(),
    nn.Linear(128, 10)
)
seed = BaseEvolvableModule(seed_layer)
seed.dynamic_id = "seed_cortex"
seed.set_specialty("general")  # Mark as General Core
network.add_layer(seed)

# 3. Standard PyTorch training loop — NO growth happens
optimizer = torch.optim.Adam(network.parameters(), lr=1e-3)

for epoch in range(100):
    optimizer.zero_grad()
    output = network(input_data)
    loss = criterion(output, targets)
    loss.backward()
    optimizer.step()
```

### Phase 3: Adulthood (`adaptive` Growth)

**Goal**: Adapt to new, complex tasks by growing specialized organs.  
**Behavior**: The model monitors entropy. High confusion triggers **Mitosis**.

```python
from dno import GrowthEngine

# 1. Switch Phase
config.training_phase = 'adaptive'

# 2. Initialize Growth Engine
growth_engine = GrowthEngine(network, config)

# 3. Training loop with entropy monitoring
for step in range(1000):
    optimizer.zero_grad()
    output = network(inputs)
    loss = criterion(output, targets)
    loss.backward()
    optimizer.step()
    
    # Calculate entropy manually (or use your own tracking)
    with torch.no_grad():
        probs = torch.softmax(output, dim=-1)
        entropy = -torch.sum(probs * torch.log(probs + 1e-9), dim=-1).mean().item()
    
    # Check growth trigger (pass recent entropy history)
    entropy_history.append(entropy)
    is_triggered, reason = growth_engine.check_growth_trigger(
        entropy_history=entropy_history[-10:],
        current_step=step,
        current_loss=loss.item()
    )
    
    if is_triggered:
        print(f"🌟 EPIPHANY! Reason: {reason}")
        
        # Trigger Mitosis — clone the seed into a new expert
        growth_engine.mitosis(
            parent_uuid="seed_cortex",
            optimizer=optimizer,
            specialty_tag="expert_coding_v1"
        )
    
    # Decay new layer learning rates
    growth_engine.stabilize_optimizer(optimizer)
```

### Phase 4: Fluid Serialization (Save/Load)

Standard `torch.save` fails on DNOs because the architecture changes dynamically. Use the `.dno` format.

```python
# Save everything (Topology + Weights + Config + History)
network.save_dno("my_organism.dno")

# Or save just the state_dict for standard PyTorch compatibility
network.save_standard("checkpoint.pt")
```

Loading requires a **module factory** — a function that reconstructs your custom blocks by type name:

```python
from dno import DynamicNetwork
from dno.core.blueprints import SeedBlock

def my_factory(type_name):
    """Reconstructs modules by their class name."""
    if type_name == "SeedBlock":
        return SeedBlock(input_dim=128, hidden_dim=256)
    if type_name == "Sequential":
        return nn.Sequential(nn.Linear(128, 128), nn.ReLU(), nn.Linear(128, 10))
    # Return None to trigger automatic GPTModel fallback
    return None

loaded_network = DynamicNetwork.load_dno("my_organism.dno", module_factory=my_factory)
```

> **Note**: If the factory returns `None` or raises an exception for a module type, DNO automatically reconstructs it as a `GPTModel` Transformer block. See [Zombie Repair](#robust-persistence--zombie-repair) below.

---

## 🔧 Advanced Mechanics

### Surgical Training Control

Use `GrowthEngine` to manually direct evolution — don't wait for entropy triggers.

```python
from dno import GrowthEngine

growth_engine = GrowthEngine(network, config)

# 1. Inject a new expert lobe (forced mitosis)
# IMPORTANT: Pass the optimizer so its parameters are registered!
growth_engine.inject_layer(
    parent_tag_or_uuid="seed_cortex",
    new_tag="expert_math_v1",
    optimizer=optimizer,
    base_lr=0.001
)

# 2. Freeze everything EXCEPT the new expert
# This prevents interference during focused training
growth_engine.freeze_all_except("expert_math_v1")

# 3. Train on math dataset...
for batch in math_dataloader:
    optimizer.zero_grad()
    output = network(batch)
    loss = criterion(output, targets)
    loss.backward()
    optimizer.step()
```

### Robust Persistence & Zombie Repair

DNO is resilient to corruption and missing class definitions. If you load a brain but the factory can't reconstruct a specific layer:

1. **Automatic Fallback**: The broken layer is replaced with a generic `GPTModel` (Transformer Decoder block).
2. **Deep Scan & Repair**: After loading, a full scan identifies and replaces any `None` or broken modules.
3. **Topology Preservation**: All graph connections (edges, inputs, outputs) remain intact.

```python
# Even with a broken/incomplete factory, loading succeeds
net = DynamicNetwork.load_dno("brain.dno", module_factory=lambda t: None)
# >> [WARNING] Factory failed for SeedBlock...
# >> [REPAIR] Reconstructing as default GPTModel.
# >> [OK] Organism Resurrected.
```

### Dynamic Gradient Locking 🔒

DNO automatically manages `requires_grad` to prevent **Catastrophic Forgetting**:

| Data Type | General Core | Expert Lobes |
|-----------|:------------:|:------------:|
| **Familiar (CPT)** | ✅ Training | ❄️ Frozen |
| **Novel (SFT)** | ❄️ Frozen | ✅ Training |
| **Scratch Phase** | ✅ Training | N/A (no experts yet) |

This happens automatically inside `network.forward()` based on entropy-based novelty detection.

### Auto-Casting 🛡️

If you pass raw `LongTensor` inputs (Token IDs) to a module expecting floats:

- DNO performs a **Deep Scan** of all submodules to check for `nn.Embedding` layers.
- If an `Embedding` is found → data is preserved as `Long` (correct behavior).
- If no `Embedding` → data is auto-cast to `Float32` to prevent `dtype mismatch` errors.

### Survival Engine (Pruning) 🧹

The `SurvivalEngine` acts as the organism's immune system:

```python
from dno import SurvivalEngine

survival = SurvivalEngine(manager, config)

# Inside training loop:
# 1. Monitor utility (Information Gain via KL Divergence)
for uuid, module in manager.registry.items():
    survival.calculate_information_gain(module, input_tensor, output_tensor)

# 2. Decay dying layers' weights
survival.apply_selective_decay(optimizer)

# 3. Garbage collect dead tissue
removed = survival.garbage_collect(network)
print(f"Pruned {removed} dead layers")
```

Layers with low utility score enter a **grace period**. If they don't recover, their weights decay to zero and they're removed from the graph.

### Expert Affinity Routing 🎯

**v0.5.0** — Instead of training ALL experts on every data type, DNO now automatically discovers which expert best understands the incoming data.

```
  New Data Arrives
       |
       v
  ┌─────────────────────────────────┐
  │  AFFINITY PROBING               │
  │                                 │
  │  Test Expert 1 → Entropy: 2.1  │ ← Best match!
  │  Test Expert 2 → Entropy: 4.8  │
  │  Test Expert 3 → Entropy: 3.5  │
  └────────────────┬────────────────┘
                   |
         ┌─────────┴──────────┐
         │ Best < Threshold?  │
         ├─── YES ────────────┤
         │ Train Expert 1     │
         │ Freeze all others  │
         ├─── NO ─────────────┤
         │ Create NEW Expert  │
         │ via Mitosis        │
         └────────────────────┘
```

**Usage:**

```python
from dno import GrowthEngine

growth_engine = GrowthEngine(network, config)

# Collect some probe data (first N batches)
probe_data = [batch for i, batch in enumerate(dataloader) if i < 10]

# Find the best expert for this data
best_expert_id, results = growth_engine.probe_expert_affinity(probe_data)

if best_expert_id:
    # Expert found! Only that expert will train.
    # network._active_expert_id is already set.
    print(f"Training expert: {best_expert_id}")
else:
    # No expert fits — create a new one
    growth_engine.mitosis(
        parent_uuid="seed_cortex",
        optimizer=optimizer,
        specialty_tag="new_domain_expert"
    )
```

**Selective Gradient Locking with Affinity:**

| Data Type | Core | Active Expert | Other Experts |
|-----------|:----:|:-------------:|:-------------:|
| **SFT + Affinity** | ❄️ Frozen | ✅ Training | ❄️ Frozen |
| **SFT (no affinity)** | ❄️ Frozen | ✅ Training | ✅ Training |
| **CPT** | ✅ Training | ❄️ Frozen | ❄️ Frozen |

### Adaptive Entropy Growth 📈

**v0.4.0** — Instead of a fixed entropy threshold, DNO establishes a **baseline** from the data and only triggers growth on genuine **spikes**.

```python
config = DnoConfig(
    training_phase='adaptive',
    entropy_baseline_window=50,   # Steps to establish baseline
    entropy_spike_factor=1.3,     # Growth if entropy > baseline × 1.3
    evolution_cooldown_steps=200,  # Min steps between growths
)
```

```
  Step 1-50:    WARMUP → Entropy baseline = 3.0
  Step 51-199:  Monitor → Entropy ~3.1 → No spike → No growth
  Step 200:     Entropy 4.2 > 3.9 (3.0×1.3) → SPIKE → Mitosis!
  Step 201-250: NEW WARMUP → New baseline for new expert
```

---

## 📖 API Reference

### Core Classes

| Class | Module | Description |
|-------|--------|-------------|
| `DnoConfig` | `dno.config` | Dataclass with all organism parameters (growth, survival, hardware limits) |
| `OrganismManager` | `dno.core.manager` | Central graph manager — tracks nodes, edges, and topology |
| `DynamicNetwork` | `dno.core.network` | The executable body — runs forward pass through the organism graph |
| `BaseEvolvableModule` | `dno.core.base` | Wrapper that adds biological metadata and energy tracking to any `nn.Module` |
| `GrowthEngine` | `dno.core.growth` | Handles mitosis, entropy triggers, optimizer sync, and surgical controls |
| `SurvivalEngine` | `dno.core.survival` | Monitors utility, applies weight decay, garbage collects dead layers |

### Blueprint Classes

| Class | Module | Description |
|-------|--------|-------------|
| `SeedBlock` | `dno.core.blueprints` | Specialized block with genetic masking (partial layer freezing) |
| `IdentityWrapper` | `dno.core.blueprints` | Gamma-gated residual wrapper — new layers enter as identity ops (`gamma=0`) |
| `GPTModel` | `dno.core.blueprints` | Standard Transformer Decoder block — used as default fallback during load |

### Key Methods

| Method | Class | Description |
|--------|-------|-------------|
| `add_layer(module, parents)` | `DynamicNetwork` | Adds a layer to the organism graph |
| `save_dno(path)` | `DynamicNetwork` | Saves topology + weights + config as `.dno` zip |
| `save_standard(path)` | `DynamicNetwork` | Saves only the `state_dict` (`.pt` format) |
| `load_dno(path, factory)` | `DynamicNetwork` | Class method — loads `.dno` with auto-repair |
| `mitosis(parent, optimizer, ...)` | `GrowthEngine` | Clones a layer with smart noise and rewiring |
| `inject_layer(parent, tag, optimizer)` | `GrowthEngine` | Manual mitosis trigger |
| `freeze_all_except(tag)` | `GrowthEngine` | Freezes all modules except those matching `tag` |
| `check_growth_trigger(entropy, step)` | `GrowthEngine` | Returns `(bool, reason)` for growth decision |
| `probe_expert_affinity(data_batches)` | `GrowthEngine` | Tests each expert, returns `(best_id, results)` |
| `set_active_expert(expert_id)` | `DynamicNetwork` | Activate only this expert (+ core) |
| `clear_active_expert()` | `DynamicNetwork` | Reactivate all experts |
| `get_module(tag_or_uuid)` | `OrganismManager` | Lookup by specialty tag or UUID |
| `list_organs()` | `OrganismManager` | Returns summary of all active layers |
| `garbage_collect(network)` | `SurvivalEngine` | Removes dead layers from graph |

### Utility Functions

| Function | Module | Description |
|----------|--------|-------------|
| `load_dno(path, factory)` | `dno` | Top-level shortcut for `DynamicNetwork.load_dno` |
| `print_organism_status(manager)` | `dno.utils.dashboard` | Pretty-prints organism anatomy to terminal |

---

## 📋 Changelog

### v0.5.0 — Expert Affinity Routing
- **New**: `probe_expert_affinity()` — tests each expert to find the best match for new data
- **New**: `set_active_expert()` / `clear_active_expert()` — selective expert activation
- **New**: Selective gradient locking — only the matched expert trains
- **New**: Forward pass filtering — non-active experts skipped for isolated probing
- **New**: `affinity_probe_steps` config parameter

### v0.4.x — Adaptive Entropy Growth
- **New**: Baseline + spike detection replaces fixed entropy threshold
- **New**: `entropy_baseline_window`, `entropy_spike_factor` config parameters
- **New**: Post-growth baseline reset for new expert warmup
- **Fixed**: Cooldown now properly resets after each mitosis event
- **Fixed**: Explicit `_growth_locked_until` lock prevents rapid sequential mitosis
- **New**: Debug logging for growth trigger blocking reasons

### v0.3.x — Bug Fixes & Stability
- **Fixed**: Gradient locking bug when no experts exist
- **Fixed**: Default `vram_limit_gb` raised from 4.0 to 12.0
- **Fixed**: Cooldown reset after mitosis

### v0.3.0 — Project Cleanup & Definitive API
- **Breaking**: Removed stale `organism.py` (use `manager.py` / `OrganismManager`)
- **New**: All public classes exported from top-level `dno` package
- **New**: `MANIFEST.in` ensures README/LICENSE in distributions
- **Fixed**: `train_demo.py` updated with correct imports
- **Docs**: Complete README rewrite with Quickstart, Architecture, API Reference, and Changelog

### v0.2.7 — Surgical Training Control
- **New**: `GrowthEngine.inject_layer()` — manual mitosis trigger
- **New**: `GrowthEngine.freeze_all_except()` — surgical freeze
- **New**: `OrganismManager.get_module()` — tag-based lookup
- **New**: `OrganismManager.list_organs()` — organ summary
- **New**: `GPTModel` blueprint — Transformer Decoder fallback
- **New**: Robust `load_dno()` with automatic Zombie Repair
- **New**: Deep Scan & Repair on load

### v0.2.6 — Auto-Casting
- **New**: Deep scan for nested `nn.Embedding` layers during auto-cast
- **Fixed**: `LongTensor` correctly preserved for Embedding inputs

### v0.2.0 — Specialized Organisms
- **New**: Data-Aware Routing (CPT vs SFT)
- **New**: Dynamic Gradient Locking
- **New**: `set_specialty()` for tagging modules
- **New**: Smart Aggregation for parallel branches

### v0.1.x — Foundation
- Core `OrganismManager` graph engine
- `BaseEvolvableModule` with energy tracking
- `GrowthEngine` with entropy-based mitosis
- `SurvivalEngine` with KL-Divergence utility monitoring
- `IdentityWrapper` with gamma gating
- `SeedBlock` with genetic masking
- Fluid Serialization (`.dno` format)
- `SurvivalEngine` garbage collection
- Dashboard visualization

---

## 🤝 Contributing

DNO is an open-source experiment. We welcome contributions!

- **Bug Reports**: Open an issue on GitHub
- **Feature Ideas**: New "Organs" (Memory modules, Attention variants, RL-based growth triggers)
- **Pull Requests**: Fork, implement, test, and submit!

## 📜 License

MIT License. See [LICENSE](LICENSE) for details.