binxx71/modern-transformer-1.5b

Modern Transformer 1.5B - GGUF Release

This repository contains the GGUF binary release of the custom 1.5-billion parameter modern transformer developed by Binyam, featuring Grouped-Query Attention (GQA), SwiGLU FFN, RMSNorm, and Rotary Position Embeddings (RoPE), fully aligned via SFT and GRPO[cite: 1].

Files Included

• modern_transformer_1.5b_aligned.gguf: The standalone binary GGUF weights container optimized for local execution.

Architectural Specifications

• Parameters: 1.5 Billion (28 Transformer Blocks)
• Hidden Dimension ($d_{model}$): 2048
• Attention Heads: 16 Query Heads / 4 KV Heads (GQA)
• Feed-Forward Dimension ($d_{ff}$): 6144 (SwiGLU)
• Normalization: RMSNorm ($\epsilon = 1 imes10^{-6}$)