VarunHujan/Karna-49M-SFT-Model

📖 Karna-49M Model (GGUF Formats)

Karna-49M is a hyper-optimized 49.38 Million parameter Small Language Model (SLM) engineered for long-form narrative generation, character-anchored dialogues, and multi-genre creative writing.

Trained from scratch and fine-tuned using Supervised Fine-Tuning (SFT) with ChatML format, it achieves fluent, coherent storytelling with instant Time-To-First-Token (<100ms) and ~35+ tokens/second on standard laptop CPUs.

---

📦 Available GGUF Quantizations

File Name	Format	Size	Description
karna_49m_sft_f16.gguf	FP16	~145.8 MB	Gold Master: Full 16-bit precision, highest fidelity, zero quantization loss.
karna_49m_sft_q8_0.gguf	Q8_0	~78.4 MB	Recommended: 8-bit quantization with near-lossless perplexity.
karna_49m_sft_q4_0.gguf	Q4_0	~42.4 MB	Ultra-Lightweight: 4-bit quantization, fits anywhere, fast on micro-devices.

---

🧠 Architecture Specification

• Parameters: 49,384,384 (~49.38M)
• Backbone: LLaMA-style Transformer Decoder
• Attention: Grouped-Query Attention (GQA 8 Query Heads : 4 KV Heads)
• Context Window: 2,048 tokens
• Hidden Dimension: 448
• Layers: 12
• Feed-Forward: SwiGLU ($d_{ffn} = 1,216$)
• Positional Encoding: RoPE ($\theta = 10,000.0$)
• Normalization: RMSNorm ($\epsilon = 10^{-5}$)
• Vocabulary: 50,304 tokens (GPT-2 BPE + ChatML Special Control Tokens)

---

🚀 How to Run

1. With llama.cpp

# Run interactive chat
./llama-cli -m karna_49m_sft_f16.gguf -p "<|im_start|>user
Write a story about a dragon guarding a hidden garden.<|im_end|>
<|im_start|>assistant
" -n 256 --temp 0.7

2. With Ollama

Create a file named Modelfile:

FROM ./karna_49m_sft_f16.gguf

TEMPLATE """<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER temperature 0.7
PARAMETER top_k 40
PARAMETER repeat_penalty 1.2
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"

Then create and run:

ollama create karna-49m -f Modelfile
ollama run karna-49m "Write a story about an astronomer who discovered a singing star."

### 3. With LM Studio
Simply drag and drop any `.gguf` file into LM Studio's local models folder.

---

## 🎭 Prompt Format (ChatML)

```text
<|im_start|>user
Write a short story about an old clockmaker in Prague.<|im_end|>
<|im_start|>assistant

---

📄 License

MIT License