ak-lm-small

Tiny pretrained Malayalam language model (75M parameters)

AK-LM-SMALL (Akshara Language Model Small) is a 75M small Malayalam GPT style Model ,Build from Scratch

Developed by EnduraSolution.

Overview

• GPT-style decoder-only Transformer
• 75 Million parameters
• Custom Malayalam SentencePiece tokenizer
• Vocabulary: 24,000
• Context length: 256

Training Data

• IndicCorp Malayalam
• Malayalam Wikipedia
• Kerala-focused corpus
• Small English Wikipedia subset

Intended Uses

• Malayalam text generation
• Text completion
• Research
• Fine-tuning

Limitations

This is a pretrained base model.

It is not instruction tuned.

Example

from transformers import GPT2LMHeadModel
import sentencepiece as spm

sp = spm.SentencePieceProcessor(model_file='ml_sp.model')
model = GPT2LMHeadModel.from_pretrained('endurasolution/ak-lm-small')

License

Apache-2.0

---

Made with ❤️ by EnduraSolution