Rage AI · rage-gen-1

The AI agent programming model, built from scratch — no pretrained starting point, no borrowed weights: tokenizer, architecture, trainer and data pipeline all built for this model. Developed by DevNameGelo, powered by the RGC (Rage Gen Core) engine.

🔥 gen is the capability level: rage-gen-1 is the current generation; rage-gen-2 is the planned upgrade.

Specs — target vs. current (read this, it matters)

	Target specification (roadmap)	Free-tier build (this release)
Parameters	30B	nano ~1.6M → t4 ~74M (default Colab build) → a100 ~150M
Context window	300k	1024 (t4 build; architecture is window-scalable)
Max output	64k	bounded by context window in current builds
Modalities	text + vision	text (vision adapter code shipped, training is roadmap)
Streaming	✅	✅ token-by-token generator
Reasoning	✅ low/high	✅ explicit think-block training, prompt-level control
Tool calling	✅	✅ JSON protocol + demo tools (calculator, get_time, echo)
Languages	multilingual, English-first	✅ 15 languages
Knowledge cutoff	September 2026	September 2026

Honest framing: the 30B/300k/64k spec defines where rage-gen-1 is headed; training 30B parameters needs data-center accelerators far beyond a free Colab T4. What ships here is the full working stack — architecture, tokenizer, trainer, dataset, Space — proven end-to-end at the t4 scale (~74M) and ready to scale.

Architecture (RGC engine)

Decoder-only transformer, everything implemented from scratch in modeling_rage_gen.py:

• RMSNorm pre-norm blocks
• RoPE rotary position embeddings (fp32 tables, applied to Q/K)
• GQA grouped-query attention (12 Q heads / 4 KV heads on t4 build) — smaller KV cache, faster inference
• SwiGLU feed-forward (≈ 8/3× width)
• Fused KV-cache incremental decoding + causal masks via scaled_dot_product_attention
• Byte-level tokenizer (265 tokens: 256 bytes + 9 specials) — zero unknowns, perfect round-trip on any UTF-8

Chat format is ChatML-style with dedicated zones:

<|im_start|>user
question<|im_end|>
<|im_start|>assistant
<|think_start|>
(reasoning chain: short for low, step-by-step for high)
<|think_end|>
answer<|im_end|>

Tool calls:

<|im_start|>assistant
<|tool_call|>{"name": "calculator", "arguments": {"expression": "12 * 9"}}<|im_end|>
<|im_start|>tool
<|tool_response|>108<|im_end|>
<|im_start|>assistant
12 * 9 = 108.<|im_end|>

Training on Colab (free T4)

Open colab_training.ipynb in Colab → Runtime type: T4 GPU → Run all. The notebook:

1. Installs deps and logs into Hugging Face (token via getpass/secrets — never hardcoded)
2. Pulls the code (this repo) and the dataset (rgcmainhub/rage-gen-1-dataset)
3. Trains with the RGC trainer: python train.py --config t4 --steps 3000 --max-seconds 7500 --push
   • fp16 autocast (T4 has no bf16), batch 24 × seq 1024, ~40–60 min on T4
   • graceful time-budget stop, resumable checkpoints, train_log.jsonl metrics
4. Streams a quick self-test conversation
5. Pushes weights (model.safetensors) back to this repo → the demo Space goes live

Usage

from modeling_rage_gen import RageGen1
from rage_tokenizer import RageTokenizer

model = RageGen1.load("model.safetensors")      # + model.config.json side-car
tok = RageTokenizer.load("tokenizer.json")

CLI chat (streaming, reasoning modes, tool loop):

python inference.py --from-hub rgcmainhub/rage-gen-1
# in the REPL:  /think high

Demo Space

rgcmainhub/rage-ai — Gradio chat with streaming. It pulls weights from this repo at startup; before the first Colab training run completes it displays a friendly "training not done yet" notice instead of failing.

Files

File	What it is
modeling_rage_gen.py	the transformer (RGC engine core)
rage_tokenizer.py	byte-level tokenizer + chat template
multimodal.py	vision adapter (roadmap module, untrained)
train.py	RGC trainer (AMP, resume, time budget, HF push)
inference.py	streaming CLI chat + reasoning + tool loop
data/build_corpus.py	synthetic corpus generator
spec.json	full spec sheet (target + build levels)
colab_training.ipynb	the Colab training notebook

Dataset

rgcmainhub/rage-gen-1-dataset — ~17.8k examples / ~9MB across identity, chat, coding (Next.js/HTML/Three.js/Python/Luau/Java/C++), reasoning (low/high chains), tool calls, agent tasks, knowledge and 15 languages.

License

MIT — © DevNameGelo. Model, code, dataset and Space all MIT.