SLM-125M Causal Language Model
A custom 125M parameter causal language model trained from scratch on legal opinions, SEC filings, and educational web data.
Model Details
- Architecture: Llama-based (12 layers, 12 attention heads, hidden dim 768)
- Vocabulary Size: 16,384
- Parameters: 125.8M
- Training Tokens: ~2 Billion tokens (SEC filings, US Case-Law, and FineWeb-Edu)
- Downloads last month
- 10