Frox AI Morph Nano
A 207M parameter language model trained from scratch on WikiText-2.
Built with the Frox Morph 1.1 architecture (RMSNorm, SwiGLU, RoPE, GQA).
Architecture
- Hidden: 1024 | Layers: 12 | Heads: 8/4 (GQA)
- Context: 4096 tokens | Vocab: 64K BPE
- Params: 207M
