🌲 Pine 6.0 (1.225 Billion Parameters)

Built 100% from mathematical scratch in raw PyTorch by Se62em & Antigravity.

Parameters: 1,224,837,120 (~1.225 Billion)
Architecture: Decoder-Only Transformer (24 Layers, 2048 Dim, 16 Heads)
Positional Embeddings: Rotary Position Embeddings (RoPE)
Activations & Norm: SwiGLU / RMSNorm
Tokenizer: Byte-Level BPE (8,192 Vocabulary)
Alignment: 4-Stage Pipeline (FineWeb-Edu Pre-training ➔ Balanced SFT ➔ DPO Anti-Hallucination ➔ Premise Distractor Logic)

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support