ParallaxOpen/Vela-15M-Preview

Vela-Lumen-15M

A 15M-parameter math content generator trained from scratch on a single 8GB laptop GPU.

Note: This is a custom architecture. Use the provided GGUF files or inference.py for inference.

Downloads

Format	File	Size	How to Use
GGUF F16	vela-lumen-15m-f16.gguf	16 MB	Recommended — Works with Ollama, llama.cpp, LM Studio
Safetensors FP16	model-fp16.safetensors	15 MB	Half precision, Python
Safetensors FP32	model.safetensors	31 MB	Full precision, training

Quick Start (Ollama)

wget https://huggingface.co/ParallaxOpen/Vela-Lumen-15M/resolve/main/vela-lumen-15m-f16.gguf
cat > Modelfile << 'EOF'
FROM vela-lumen-15m-f16.gguf
TEMPLATE "{{ .System }}{{ .Prompt }}"
SYSTEM "You are a math tutor."
EOF
ollama create vela-lumen-15m -f Modelfile
ollama run vela-lumen-15m

Capabilities

What it CAN do:

• Generate math word problems and solutions
• Generate math tutorials and explanations
• Text completion for math topics
• Attempt to answer simple questions

What it CAN'T do:

• Solve arithmetic reliably (0% GSM8K)
• Complex reasoning tasks
• Long-form coherent text

Best for:

• Math content generation
• Educational content creation
• Practice problem generation
• Text completion for math topics

Model Details

Property	Value
Developer	ParallaxOpen
Parameters	15,486,080
Architecture	Transformer Decoder, GQA 4:2, SwiGLU, RMSNorm, RoPE
Context	512 tokens
Vocab	24,189
License	CC BY-NC 4.0

Training

• Pretrain: 200K steps on 7B tokens (1.5B web + 5.6B math)
• SFT: 3K steps on 14K math problems
• Total: ~10 hours on RTX 5060

Architecture

Component	Value
Layers	6
Hidden dim	320
Q heads	4
KV heads	2 (GQA)
FFN dim	1,024 (SwiGLU)
Norm	RMSNorm
Position	RoPE (theta=10,000)
Weight tying	Yes

Citation

@software{vela-lumen-15m,
  title={Vela-Lumen-15M: A 15M-param Math Content Generator},
  author={ParallaxOpen},
  year={2026},
  note={Trained on a single RTX 5060 Laptop GPU in 10 hours}
}