jacmor64/ares-seed-30m

Ares Seed 30M

This is the first ~30M-parameter Ares proof checkpoint trained from scratch. It uses the official configs/ares_30m.json architecture and generates from its own weights.

Training summary

• Architecture: Ares decoder-only Transformer
• Parameters: about 28.3M
• Context: 512 tokens
• Tokenizer: BPE trained from scratch, 8192 vocab
• Training: 1,500 CPU steps
• Data mixture:
  • Simple Wikipedia streamed from wikimedia/wikipedia
  • Ares machine-learning process curriculum
  • Ares roleplay curriculum
• Best validation loss: about 4.634 at step 1249

Comparison with smaller seed checkpoints

• Ares Seed 1M: about 1.05M params, best validation loss around 4.42 after 5,000 steps.
• Ares Seed 6M: about 6.23M params, best validation loss around 3.92 after 2,500 steps.
• Ares Seed 30M: about 28.3M params, best validation loss around 4.63 after 1,500 CPU steps.

The 30M model has more capacity, but this small seed corpus and short CPU run are not enough for it to outperform the 6M model yet. It needs the full Colab/GPU notebook with more Wikipedia and roleplay data.

Files

• ares_seed_30m_inference.pt — slim PyTorch checkpoint containing model weights/config.
• tokenizer.json — BPE tokenizer.
• config.json — Ares 30M model config.
• sample_generation.txt — sample outputs.
• training_report.html — offline training report.
• training_manifest.json — data mixture and training summary.

Use

Clone the Ares Static Lab Space repo and run:

python -m ares_core.generate \
  --checkpoint ares_seed_30m_inference.pt \
  --tokenizer tokenizer.json \
  --prompt "Roleplay as a machine-learning tutor" \
  --device cpu

Caveat

This is a parameter-scaling proof checkpoint, not a capable assistant. For useful quality, run colab/Ares_Wikipedia_Roleplay_Brain_Training.ipynb on GPU with the larger dataset settings.