Kavya-1-7B

A 7B parameter Telugu songwriting model that composes original lyrics in traditional pallavi–charanam form.

Research release. Output captures song structure and rhythm well, but individual Telugu words are often malformed. See Known limitation before use.

---

Overview

Kavya-1-7B is a Telugu lyric-writing model. Given a theme, mood, and style, it composes complete original songs in Telugu script — structured as pallavi, anupallavi, and charanam — with the imagery, emotional register, and metrical rhythm of modern Telugu film music.

The model was trained on roughly 9,100 curated Telugu songs spanning contemporary cinema and classical devotional composition, giving it both a modern lyrical voice and a grounding in older poetic idiom. It is tuned to write new lyrics rather than reproduce existing ones.

Kavya-1 adapts a strong multilingual foundation to Telugu poetic form — a domain where general-purpose models tend to produce stilted, unsingable output that ignores metrical structure.

Kavya-1 is the first release in the Kavya series.

Quickstart

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "sainitishb/Kavya-1-7B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": "మీరు అనుభవజ్ఞుడైన తెలుగు సినీ గేయ రచయిత. సహజమైన, ఒరిజినల్, "
                   "పాడుకోదగిన తెలుగు పాటలు రాస్తారు. ఉన్న పాటలను కాపీ చేయరు.",
    },
    {
        "role": "user",
        "content": """అమ్మ ప్రేమ గురించి ఒక పూర్తిస్థాయి ఒరిజినల్ తెలుగు పాట రాయండి.
శైలి: భావోద్వేగమైన ఆధునిక తెలుగు సినిమా పాట.
రూపం:
శీర్షిక
పల్లవి
అనుపల్లవి
చరణం 1
చరణం 2
తెలుగు లిపి మాత్రమే వాడండి. పాట పూర్తిగా ముగియాలి.""",
    },
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=650,
        do_sample=True,
        temperature=0.55,
        top_p=0.88,
        repetition_penalty=1.18,
        no_repeat_ngram_size=4,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Prompting

Output quality depends heavily on prompt structure. The model responds best when the request specifies four things:

Element	Telugu	Purpose
Theme	subject of the song	what the song is about
Style (శైలి)	e.g. modern film song, devotional	sets register and vocabulary
Mood (భావం)	e.g. longing, celebration, gratitude	sets emotional tone
Structure (రూపం)	pallavi / anupallavi / charanam	requests explicit sections

Closing the prompt with an instruction to use Telugu script only and to finish the song completely (తెలుగు లిపి మాత్రమే వాడండి. పాట పూర్తిగా ముగియాలి.) measurably reduces script-mixing and truncated endings.

Recommended generation settings

Parameter	Value	Note
temperature	0.55	higher values drift off-theme
top_p	0.88
repetition_penalty	1.18	song refrains repeat naturally; too high flattens them
no_repeat_ngram_size	4	prevents looping without blocking refrains
max_new_tokens	650	enough for a full multi-charanam song

Intended use

Kavya-1-7B is built for creative work:

• Drafting and ideating Telugu song lyrics
• Songwriting assistance — generating alternate refrains, verses, or imagery
• Telugu-language creative writing and education
• Research into low-resource language generation and Indic poetic form

Out of scope. This model is not a source of factual information, is not suitable for any commercial, legal, medical, or safety-critical application, and should not be used to imitate a specific living writer's voice or to reproduce existing copyrighted songs.

Training

Records	9,102 curated Telugu songs
Split	8,648 train / 318 validation / 136 test
Epochs	2
Learning rate	2e-4
Effective batch size	16 (batch 4 × grad accum 4)
Max sequence length	2,048 tokens
Warmup ratio	0.03
Precision	bfloat16

Training data was assembled from publicly available collections of Telugu song lyrics, then deduplicated and filtered for length, script consistency, and completeness. Each record was formatted as an instruction–response pair pairing a structured songwriting brief with its lyrics.

The released weights are a merged, standalone model — no adapter loading required.

⚠️ Known limitation: Telugu orthography

This is a research release with a significant known defect. The base model's tokenizer contains no Telugu characters — Telugu is encoded as raw UTF-8 bytes at roughly 0.69 characters per token, versus 4.57 for English. The model must therefore assemble every Telugu character byte by byte.

As a result, generated output frequently contains malformed words and broken conjuncts that are not valid Telugu. Song structure, rhythm, and overall shape are learned well; individual word correctness is unreliable.

Treat this as an experimental artifact rather than a production model. A from-scratch successor with a Telugu-native tokenizer (4.61 chars/token) is in development.

Limitations

• Malformed output. See the known-limitation notice above. This is the dominant quality issue.
• Telugu script only. The model is trained and evaluated on Telugu script. Romanized Telugu input produces markedly worse output.
• Film-song register. Its default voice is contemporary cinema. Other registers — folk, classical, experimental — are weaker and need explicit prompting.
• Long generations drift. Beyond roughly 650 tokens, thematic coherence degrades and refrains may loop.
• Memorization risk. Like any model trained on a finite corpus, it can occasionally produce lines closely resembling training text. Check output before any public or commercial use.
• Not factual. Names, films, and events appearing in output are frequently invented.
• Unaudited biases. The training corpus reflects the themes and gender portrayals common to Telugu film lyrics, including their stereotypes. No bias evaluation has been performed.

License and attribution

Released under the Apache License 2.0.

This model is a fine-tune of Qwen/Qwen2.5-7B-Instruct, which is itself Apache-2.0 licensed. The base weights were modified through low-rank fine-tuning on Telugu song lyrics and merged into the released checkpoint.

Training material was drawn from publicly available lyric collections. Lyrics may carry rights held by their original authors and publishers; users are responsible for ensuring their use of generated output complies with applicable copyright law in their jurisdiction.

Citation

@misc{kavya1_7b,
  title  = {Kavya-1-7B: A Telugu Songwriting Model},
  author = {sainitishb},
  year   = {2026},
  url    = {https://huggingface.co/sainitishb/Kavya-1-7B}
}