sweta-125m-multilingual

Sweta-125M is a decoder-only multilingual language model trained from scratch.

Model Details

• Repository: shreyash-pandey-katni/sweta-125m-multilingual
• Checkpoint: best.pt (step 2250)
• Parameters: 134.11M
• Languages: English, Hindi, Kannada
• Architecture: RoPE + SwiGLU + RMSNorm decoder-only Transformer

Evaluation Results

Evaluated with evaluate_multilingual.py using saved artifacts from this training run.

• Perplexity (Hindi): 14.4639
• Perplexity (Kannada): 33.9995
• English perplexity: not reported in this run (no dedicated English eval corpus available in deduped paths).

Generation Samples

English prompt: The Government of India announced

The Government of India announced new of plan development The of, on 16 2013
The Minister Transport had the of the to the of, Minister Minister, Minister State, the Minister State,,,,,,,,,,,,, Minister Environment and,, Minister, and,, Minister and,
 Director of, and,,, SecretaryGeneral the Minister,,,,,, Minister and
 of, Minister,,,,,,,,,,,,,,,,,,
,,,,,,,,,,,,,,,,, Minister
,

Hindi prompt: भारत सरकार ने घोषणा की कि

भारत सरकार ने घोषणा की किंााेाे्ा क वसराोिो् सनब क पर्ाोें्ा्िाो क सव
््ि्ाी
ा।उनहलगरड सव मकषठ
ािाा्िो्ेा् मरतरमरहा।मरजगरबगटलागआमब
िाीुाैे्ाािो्ोा
ोाेाोो्
ौिा्

Kannada prompt: ಭಾರತ ಸರ್ಕಾರ ಘೋಷಿಸಿದೆ कि

ಭಾರತ ಸರ್ಕಾರ ಘೋಷಿಸಿದೆ किा, ಕಗರನಟ ಮಲನನದದೊೆಾ್ಾಿೆ್್್್ೇ್ಿ್ಾ್ುಾ್ಾ್ಾೆೈ್ಿಿ್ಿಾಾಿಂಿ್ುು್್್ಿ್್್ು ವರವದಕಸಪರಧತ ಬಬಲಗಡ ಕಲಸತತರೇ್ುಾಂಿಿ್್ುಮತತತತಪರಯತಯನನಪತ |ೇೇಾೇಾಂ್ು್ಾಾಿಾೇಿ್ಾ

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "shreyash-pandey-katni/sweta-125m-multilingual"

tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "भारत सरकार ने घोषणा की कि"
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=120)
print(tok.decode(out[0], skip_special_tokens=True))