Vela 15M Preview

A 15M-parameter math content generator trained from scratch on a single 8GB laptop GPU.

Preview: This is an early version. More training data and better results coming soon.

Downloads

Format File Size How to Use
GGUF F16 vela-lumen-15m-f16.gguf 59 MB Works with Ollama, llama.cpp, LM Studio
Safetensors FP16 model-fp16.safetensors 44 MB Half precision, Python
Safetensors FP32 model.safetensors 89 MB Full precision, training

Quick Start (Ollama)

wget https://huggingface.co/ParallaxOpen/Vela-Lumen-15M/resolve/main/vela-lumen-15m-f16.gguf
cat > Modelfile << EOF
FROM vela-lumen-15m-f16.gguf
TEMPLATE "{{ .System }}{{ .Prompt }}"
SYSTEM "You are a math tutor."
EOF
ollama create vela-15m-preview -f Modelfile
ollama run vela-15m-preview

What It Does

Math Content Generation:

  • Generates math word problems and equations
  • Generates homework-style questions
  • Generates math explanations and tutorials
  • Generates LaTeX formulas

Example:

Prompt: "Create a math problem about:"
Output: "The expression (x/m * x) - (x/m * x) = x.
        Recall that (x/m * x) - (x/m * x) = x.
        Therefore, we can solve for..."

ELO Rating

Metric Value
Estimated ELO ~800-850
GSM8K 0%
Params 15.5M

Comparison

Model Params GSM8K ELO
Claude Fable 5 Unknown Unknown 1507
Llama 3.2 1B 1.26B 7.6% 1111
SmolLM2-135M 135M 1.4% N/A
Vela 15M Preview 15.5M 0% ~800-850

Model Details

Property Value
Developer ParallaxOpen
Parameters 15,486,080
Architecture Transformer Decoder, GQA 4:2, SwiGLU, RMSNorm, RoPE
Context 512 tokens
Vocab 24,189
License CC BY-NC 4.0

Training

  • Pretrain: 7B tokens, 200K steps, loss 2.5, ~10h on RTX 5060
  • SFT: 14K math samples, 5K steps, loss 3.32, ~30min on RTX 5060
  • Total: ~10 hours on a single 8GB laptop GPU

What's Next

  • More pretrain data (FineMath 50B tokens)
  • Better SFT with distilled data
  • Larger model (50M+ params)
  • Improved math reasoning

Citation

@software{vela-15m-preview,
  title={Vela 15M Preview: A 15M-param Math Content Generator},
  author={ParallaxOpen},
  year={2026},
  note={Preview version - trained on a single RTX 5060 Laptop GPU}
}
Downloads last month
-
Safetensors
Model size
23.2M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support