Supernova-TeraiLLM-v1

Supernova Terai LLM — EOS V1.

A compact Nepali-focused causal conversational language model.

Architecture

• Parameters: 36,809,984
• Embedding dimension: 256
• Transformer layers: 4
• Attention heads: 8
• FFN dimension: 1024
• Context length: 256
• GELU activation
• Pre-LayerNorm
• Causal self-attention
• Tied input/output embeddings
• EOS training enabled

Tokenizer

Supernova NepaliFast V5.

The tokenizer runtime vocabulary is larger than the neural model vocabulary. The model uses its own learned vocabulary space.

Training dataset

Supernova11c/Supernova-teraillm

• Train: 2,312
• Validation: 289
• Test: 289
• Split seed: 20261003

Training

• Optimizer: AdamW
• Learning rate: 3e-4
• Weight decay: 0.1
• Gradient clipping: 1.0
• Batch size: 16
• Maximum sequence length: 256
• Epochs completed: 3

This checkpoint is preserved as the baseline EOS model before additional behavioral-data experiments.

supernova doesn't claim that it have made a world powerful model no but it have successfully build such model that can generate Nepali text without Reliance on any other infrastructure Just Pure pytorch in just sitting in Nepal

generalisation still not achieved but generation free generation

we aim that further version will include generalisation not just Nepali test