Fractus CTE

A living AI that thinks continuously, remembers forever, and grows on its own.

What is Fractus?

Fractus is not a chatbot. It's not GPT. It's not a transformer.

Fractus is a Continuous Cognitive Agent — an AI that works like a brain, not a calculator. Instead of processing input → output in one pass, Fractus ticks like a biological system: it maintains a persistent thought state, advances it through multiple blocks of processing, remembers everything across sessions, and can grow new capacity by itself.

What makes it different from GPT/Claude?

	GPT-4 / Claude	Fractus
Thinking	One pass, done	Continuous ticks (like a heartbeat)
Memory	Forgets when context window fills	Remembers forever (survives restarts)
Learning	Retrain from scratch ($$$)	Learns from every interaction
Growth	Fixed size forever	Grows new experts at runtime
Mental states	One mode always	Shifts between cognitive modes
Where it runs	Corporate cloud	Your machine
Training	Fixed, done once	Perpetual, never stops

The 12 Building Blocks

Block	What it does
Continuous Thought Engine	The brain — thinks tick by tick through 16 blocks
Persistent Memory	Remembers you across sessions, never forgets
Cognitive Modes	Shifts mental states (focused, creative, exploratory...)
RAG Knowledge Base	Learns facts instantly — no retraining needed
Cognitive Plugins	Hot-swappable modes: analyst, coder, creative, teacher
MetaCognition	Decides its own actions: retrieve, learn, generate
Progressive Growth	Grows from 6M to 1B+ params, palier by palier
Self-Modification	Adds new experts at runtime when it needs them
PhaseRoutedMoE	Sparse experts routed by oscillator phases
Kuramoto Clock	A dynamical system that drives routing decisions
Online Trainer	Learns continuously, one chunk at a time
HF Space	Live chat demo with shared memory

Datasets (4.15 Billion Tokens)

Fractus is trained on a massive, diverse corpus available at huggingface.co/datasets/thefinalboss/fractus-datasets:

Dataset	Tokens	Content
neuro-paradigms-1b	1.78B	300 neuroscience → software architecture paradigms
neuro-code-math	~900M	Neuro-inspired coding, mathematics, algorithms
cognitive-skills	~780M	Coding, reasoning, speaking, thinking, understanding
fractus-generated-corpus	340M	Bilingual FR/EN generated by Fractus ontology engine
paradigms-full	191M	140 paradigms (neuroscience, CS, architecture)
neuro-arch-full	86M	60 neuroscience paradigms (neuro-software-architecture)
all-github-repos	54M	49 of your GitHub repos (self-aware codebase)
mega-corpus-v3	20M	Literature, philosophy, occult, masonry, science, medicine
Total	~4.15B

The corpus covers neuroscience, software architecture, philosophy, psychology, literature, esoteric traditions, programming, medicine, and Fractus's own source code.

How to Use

Install

git clone https://github.com/AFKmoney/fractus-cte.git
cd fractus-cte
pip install torch numpy tokenizers matplotlib fastapi uvicorn pydantic

Run tests

pytest tests/ -q
# → 28 passed

Train on CPU (progressive growth)

python scripts/train_progressive.py --paliers 0,1,2,3 --accumulation-steps 8

Train on GPU (1B scale)

python scripts/train_1b_gpu.py \
    --checkpoint checkpoints/fractus_palier3.pt \
    --tokens 500000000 \
    --batch-size 8 \
    --bf16 \
    --accumulation-steps 4

Use the agent

from fractus.continuous_engine import ContinuousThoughtEngine
from fractus.memory import PersistentMemory
from fractus.tokenizer import FractusTokenizer

# Build the brain
engine = ContinuousThoughtEngine(
    vocab_size=50257, d_model=128, n_heads=2, d_head=64,
    n_layers=2, n_levels=2, n_oscillators=8, coupling_rank=4,
    n_experts=4, top_k=2, expert_d_ff=128, siren_rank=32)

# Give it memory
memory = PersistentMemory(d_model=128, path="~/.fractus/memory.pt")
engine.attach_memory(memory)

# Think
engine.reset_thought(batch_size=1)
logits, confidence = engine.tick(torch.tensor([42]))
print(f"Confidence: {confidence.item():.2f}")

The Growth Path

Stage	Size	Blocks	Experts	What it can do
Palier 0	6.6M	1	4	Learn basic patterns
Palier 1	25M	2	8	Simple text generation
Palier 2	120M	4	16	Coherent fragments
Palier 3	350M	8	32	Decent text quality
Palier 4	1B	16	128	Full language model

Each stage inherits the previous one's knowledge. The model never starts from zero.

Architecture (for developers)

fractus-cte/
├── fractus/
│   ├── continuous_engine.py      ← The brain (CTE + CTEBlock)
│   │   ├── CTEBlock              One block: attention + Kuramoto + MoE
│   │   └── ContinuousThoughtEngine  Stacks N blocks, carries thought state
│   ├── memory.py                 ← Cross-session persistent memory
│   ├── cognitive_modes.py        ← Unsupervised mental state detection
│   ├── grow.py                   ← Progressive growth operator (width + depth + experts)
│   ├── rag.py                    ← Knowledge base + plugins + metacognition
│   ├── tokenizer.py              ← GPT-2 BPE tokenizer
│   ├── nn/
│   │   ├── moe.py                ← PhaseRoutedMoE (sparse, low-rank, differentiable)
│   │   ├── attention.py          ← Multi-level causal linear attention
│   │   ├── phase_ode.py          ← Kuramoto RK4 oscillators
│   │   └── lazy_siren.py         ← Low-rank weight storage
│   └── train/
│       └── online.py             ← Online trainer (SGD/AdamW, accumulation)
├── tests/                        28 tests
├── scripts/                      Training + corpus + GPU scripts
├── space/                        HF Space demo
├── docs/                         Optimization analysis
├── Fractus_White_Paper.pdf       Technical white paper v2.0
└── arxiv/                        LaTeX source for arXiv submission

Key Concepts

Tick: one step of thinking. The engine processes an observation, updates its thought state through all blocks, and optionally emits output.

Thought state: a vector that persists across ticks — the engine's "consciousness."

Chunk: 32 tokens processed in one forward pass. The thought state and per-block attention state carry between chunks.

Expert: a small neural network (low-rank W = scale·U@V^T) that specializes in certain thoughts. Only 2 of 128 active per token (sparse routing).

Kuramoto: coupled oscillators producing phase vectors that route tokens to experts. The engine's "internal clock."

Research Results (Honest)

EDT (Expert Decoupled Training): refuted. 5 variants, all ~19% worse.
Forward-Forward (Hinton 2022): refuted. Local learning can't replace global backprop.
Progressive growth: works. Warm start converges faster.
Sparse MoE low-rank: works. 2/128 experts = 64x less compute.
1345 tok/s on CPU: measured (batch=8 + SGD + all optimizations).

License

MIT. Fractus belongs to you, not to a corporation.

Author

Philippe-Antoine Robert — 2026 — rpa.tu@proton.me

thefinalboss
/

fractus-cte