Cesium2 (MORPH-AI) v6

Modular Orchestrated Reasoning with Pattern-adaptive Hot-swappable skills

A novel local AI architecture engineered for excellent reasoning and code generation while staying small enough to run on a laptop or phone and train on a free Google Colab T4.

Core design: System-1 / System-2 dual-path. A Coordinator decides how much thinking to spend and which subsystems to activate per input. Nine subsystems are wired directly into the logits so they actually train and actually change outputs:

1. Coordinator — routes between subsystems, predicts reasoning depth
2. MultiStepReasoner — iterative System-2 thinking loop (weight-tied, adaptive depth)
3. CodeAwareBias — injects code structure (indent depth, bracket balance) as a learned bias
4. ScratchpadMemory — persistent cross-turn working memory for long reasoning
5. VerifierHead — self-critique scorer for best-of-n decoding
6. Sparse MoE — top-2 of 4 experts per token: 4x capacity for ~half the compute
7. Persistent Memory — key-value memory that persists across turns
8. Skill Tokens — hot-swappable capability embeddings (no retraining for new skills)
9. Depth Embeddings — predicts task complexity and conditions on it

Quick Start

1. Upload notebooks/colab_train.ipynb and the morph-ai folder to Google Colab (free, no API key needed)
2. Run all cells (~2-3 hours on free T4 GPU)
3. Download output/morph-model/ folder
4. Install locally: pip install -r requirements.txt
5. Run: python src/runtime.py --model output/morph-model/

What Makes This NEW

Feature	Existing Models	MORPH-AI v6
Reasoning	Fixed CoT or none	System-1/2: adaptive thinking loop + MoD dynamic skip
Code awareness	Token-blind	Structure-injected (indent, brackets, AST)
Self-critique	No	Verifier best-of-n + cross-examination
Skills	Prompt engineering only	Learned embeddings, hot-swappable
Efficiency	Dense FFN everywhere	Sparse MoE + MoD + memory-efficient SDPA
Memory	Windowed context	Quantized persistent KV + scratchpad + MoD
Training	Full retrain for new skill	15min LoRA per skill + 8-bit optimizer
Multimodal	Single modality	Text + Vision + Audio + Video + Documents + Tools
Size	7B+ params	1.5B params, ~1GB quantized

Reasoning & Coding Features

• Adaptive depth: the Coordinator runs 0-4 System-2 refinement iterations per input — easy queries answer instantly, hard ones think longer. On device, low-gate inputs skip MoE+memory entirely.
• Code structure bias: per-token features (indent depth, bracket balance, code-likeness, newlines, keywords, numerics) gate a learned structure projection, so the model attends to indentation and braces.
• Self-critique: chat_best_of_n(prompt, n=4) generates 4 candidates and keeps the one the verifier scores highest — big accuracy gains on code/math at ~4x inference cost.
• Cross-turn scratchpad: reasoning state written in one turn is read back in the next, enabling multi-turn problem solving.
• Mixture of Depths: per-token gating dynamically skips transformer layers, reducing compute by 30-50% with minimal accuracy loss.
• Dynamic MoE: 4 experts with automatic pruning of underused experts during training.
• Quantized KV Cache: INT8/INT4 quantized persistent memory for memory-efficient long-context.
• Tool Use: JSON-structured function calling with built-in tools (calculator, search, code execution, time).
• Document Understanding: PDF, DOCX, and image OCR with layout-aware parsing and table extraction.
• Video Understanding: Temporal frame sampling, motion scoring, and scene change detection.
• Audio/Speech: ASR (Whisper) for transcription, TTS (Coqui/gTTS) for speech synthesis.
• Memory-efficient attention: PyTorch 2.0+ SDPA with flash attention for 30-50% memory reduction.

v6 Guardrail + Multimodal + Live-Knowledge Pipeline

The v6 pipeline wraps the trained model with deterministic, zero-model-cost layers that make execution provable and safe, adds vision/audio/video/document layers for full multimodal input, and grounds answers with live web search parsed into a knowledge graph. New v6 memory-efficient components include SDPA attention, Mixture of Depths (MoD), dynamic MoE pruning, and quantized KV cache.

Layer	File	What it does
RuntimeFSM	src/fsm.py	Whitelisted state machine: IDLE→INTAKE→GUARD_IN→VISION→AUDIO→VIDEO→DOCUMENT→SEARCH_GATE→SEARCH→FACT_EXTRACT→ROUTED→GEN→TOOL_USE→VERIFY→GUARD_OUT→RESPOND→IDLE. Illegal transitions trap to FAULT.
RuleEngine	src/rules.py + rules/rules.json	IF-THEN production rules on raw strings. Block/mask/warn output for emails, phones, harmful content, destructive shell commands.
RegexFeatureExtractor	src/regex_features.py	7-dim per-token features (code-likeness, indent, brackets, keywords, quotes, numerics) + running syntax gate (balanced brackets/quotes).
RoutingMatrix	src/routing.py + routing/routing_matrix.json	JSON skill routing with regex/keyword patterns, deterministic token indices, hot-swappable LoRA adapters.
KVStore	src/kvstore.py	Disk-backed cross-turn KV cache with TTL + LRU eviction.
VisionAnalyzer	src/vision.py	Multimodal VLM/ViT: image → visual embeddings + object detection + pixel-fact fallback (ImageFacts). Lazy model load, offline-safe.
AudioModule	src/audio.py	ASR (Whisper) + TTS (Coqui/gTTS): audio → transcription + embeddings. Lazy model load.
VideoModule	src/video.py	Temporal frame sampling + motion features + scene change detection.
DocumentModule	src/document.py	PDF/DOCX/OCR with layout-aware parsing and table extraction.
ToolRegistry	src/tools.py	JSON-structured function calling with validation and safe execution.
SearchClient + RAGPipeline	src/search.py	Keyless built-in web search — no API key, no quota. Tries DuckDuckGo → Bing → Mojeek HTML backends in order with per-backend cooldown; optional Google CSE only if keys set. Fetch → chunk → rank → packed RAG context, KV-cached.
FactExtractor + KnowledgeGraph + GraphQuery	src/facts.py	Regex NER (persons/orgs/locations/dates/numbers) + subject-relation-object triples → persistent JSON knowledge graph → queryable grounded facts.
MemoryEfficientAttention	src/architecture.py	PyTorch 2.0+ SDPA with flash attention fallback for 30-50% memory reduction.
MixtureOfDepths	src/architecture.py	Per-token dynamic layer skipping: 30-50% compute reduction with minimal accuracy loss.
DynamicMoEBlock	src/architecture.py	Sparse MoE with expert pruning: removes underused experts during training.
QuantizedMemoryModule	src/architecture.py	INT8/INT4 quantized KV cache for memory-efficient long-context memory.

src/runtime.py composes them: guardrails run before and after the model; the FSM walks _ingest (VISION → AUDIO → VIDEO → DOCUMENT → SEARCH_GATE → SEARCH → FACT_EXTRACT) to gather image + audio + video + document + live-web context; routing selects the skill adapter; best-of-n uses normalized verifier scoring with early exit; the winner is cross-examined against grounded facts (entity overlap + rule compliance); tools are executed when detected; every turn is persisted to the KV store and graph.

Live search is built in and keyless — no API key, no quota ceiling. The search backend tries DuckDuckGo → Bing → Mojeek HTML endpoints in order and auto-recovers when one is rate-limited (90s cooldown). Optional Google CSE is only used if you set GOOGLE_CSE_API_KEY + GOOGLE_CSE_ID env vars:

# optional: only needed to add Google as a 4th backend
export GOOGLE_CSE_API_KEY=your_api_key
export GOOGLE_CSE_ID=f49a9160e6e4840d2

Tests (no model needed): python tests/test_pipeline.py and python tests/test_multimodal_search.py

Project Structure

morph-ai/
├── notebooks/
│   └── colab_train.ipynb    ← Upload this + the folder to Colab
├── src/
│   ├── architecture.py      ← Model definition (16 subsystems, v6)
│   ├── train.py             ← Full training (components + LoRA + 8-bit optim)
│   ├── runtime.py           ← Local inference + skill management
│   ├── skill_generator.py   ← Free dataset generation (HF datasets)
│   ├── fsm.py               ← RuntimeFSM (v6 state machine)
│   ├── rules.py             ← RuleEngine (v6 production rules)
│   ├── regex_features.py    ← RegexFeatureExtractor (v6 token gating)
│   ├── routing.py           ← RoutingMatrix (v6 JSON skill routing)
│   ├── kvstore.py           ← KVStore (v6 persistent KV cache)
│   ├── vision.py            ← VisionAnalyzer (v6 VLM/ViT image analysis)
│   ├── audio.py             ← AudioModule (v6 Whisper ASR + Coqui TTS)
│   ├── video.py             ← VideoModule (v6 frame sampling + motion)
│   ├── document.py          ← DocumentModule (v6 PDF/DOCX/OCR)
│   ├── tools.py             ← ToolRegistry (v6 function calling)
│   ├── search.py            ← SearchClient + RAGPipeline (v6 live web)
│   └── facts.py             ← FactExtractor + KnowledgeGraph + GraphQuery (v6 NER/graph)
├── rules/rules.json         ← IF-THEN guardrail rules (editable)
├── routing/routing_matrix.json ← skill routing matrix (editable)
├── docs/ARCHITECTURE.md     ← full v6 pipeline design
├── tests/                   ← offline pipeline tests (no model needed)
├── export_gguf.py           ← Mobile/laptop GGUF export
├── skills/                  ← Skill files (hot-swappable)
├── datasets/                ← Training data (generated free)
├── output/
│   └── morph-model/         ← Trained model (download from Colab)
├── requirements.txt
└── README.md

How Skills Work

Skills are .skill JSON files containing trigger patterns, a learned embedding index (hot-swapped at runtime), and example prompts. No retraining needed to install a new skill:

from src.runtime import MorphRuntime
rt = MorphRuntime("output/morph-model/")
rt.install_skill("skills/code_expert.skill")
rt.chat("Write a Python function to sort a list", skill="code_expert")
# higher quality on code/math:
rt.chat_best_of_n("Debug this: ...", n=4)

Create new skills with skill_generator.py — uses free HuggingFace datasets, no API key needed.

Hardware Requirements

Device	RAM	Runs?	v6 Notes
Modern phone	4GB+	Yes (GGUF q4, llama.cpp/Termux)	MoD + quantized KV enable this
Laptop	8GB+	Yes (GGUF q4 or 4-bit PyTorch)	Memory-efficient SDPA + MoD
Desktop	16GB+	Yes (full precision)	Full v6 with all experts active
Raspberry Pi 5	8GB+	Yes (GGUF q2 + MoD)	MoD + expert pruning critical

Training on Colab (FREE)

What you need:

• Google account (free)
• No API keys, no payment method
• 2-3 hours of runtime (free T4 sessions are ~12hr)

Steps:

1. Go to colab.research.google.com
2. New notebook → Runtime → Change runtime type → GPU (T4)
3. Upload the notebook and the morph-ai folder (the notebook auto-chdirs to /content/morph-ai)
4. Run all cells
5. Download output/ folder when done

Free tier limits:

• T4 GPU: 16GB VRAM (enough for this model + all 16 v6 subsystems)
• Session: ~12 hours (training takes ~3 hours)
• No usage cap on free tier

v6 memory-efficient training:

python src/train.py \
  --base Qwen/Qwen2.5-1.5B-Instruct \
  --data ./datasets \
  --epochs 3 \
  --batch 4 \
  --grad-accum 8 \
  --no-4bit          # omit for 4-bit QLoRA (default on)
  --no-8bit-optim    # omit for 8-bit paged AdamW (default on)
  --prune-every 500  # prune MoE experts every 500 steps
  --mod-sparsity 0.01 # MoD sparsity loss weight

License

MIT — do whatever you want with it.