jacmor64/ares-seed-6m

Ares Seed 6M

This is a larger proof-of-training checkpoint for Ares. It is trained from scratch and generates from its own weights. It is still not a capable assistant, but it is a parameter increase over jacmor64/ares-seed-1m and validates the scaling path.

Training summary

• Architecture: Ares decoder-only Transformer
• Parameters: about 6.23M
• Context: 128 tokens
• Tokenizer: BPE trained from scratch / shared seed tokenizer, 2048 vocab
• Training: 2,500 CPU steps
• Data mixture:
  • Simple Wikipedia streamed from wikimedia/wikipedia
  • Ares machine-learning process curriculum
  • Ares roleplay curriculum
• Best validation loss: about 3.918 at step 1749

Compared with Ares Seed 1M

• Seed 1M: about 1.05M params, best validation loss around 4.42 after 5,000 steps.
• Seed 6M: about 6.23M params, best validation loss around 3.92 after 2,500 steps.

Files

• ares_seed_6m_inference.pt — slim PyTorch checkpoint containing model weights/config.
• tokenizer.json — BPE tokenizer.
• config.json — Ares model config.
• sample_generation.txt — sample outputs.
• training_report.html — offline training report.
• training_manifest.json — data mixture and training summary.

Use

Clone the Ares Static Lab Space repo and run:

python -m ares_core.generate \
  --checkpoint ares_seed_6m_inference.pt \
  --tokenizer tokenizer.json \
  --prompt "Roleplay as a machine-learning tutor" \
  --device cpu

Caveat

This checkpoint is tiny and trained on a small seed corpus. It overfits if trained too long, and outputs remain rough/repetitive. Use the Colab brain notebook in the Ares Static Lab to train the stronger 30M+ model on a larger Wikipedia/roleplay mixture.