Instructions to use Dhanush444/Telugu-lora-adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Dhanush444/Telugu-lora-adapter with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Meta-Llama-3.1-8B-Instruct-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "Dhanush444/Telugu-lora-adapter")

Transformers

How to use Dhanush444/Telugu-lora-adapter with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Dhanush444/Telugu-lora-adapter")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Dhanush444/Telugu-lora-adapter", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Dhanush444/Telugu-lora-adapter with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Dhanush444/Telugu-lora-adapter"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dhanush444/Telugu-lora-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Dhanush444/Telugu-lora-adapter

SGLang

How to use Dhanush444/Telugu-lora-adapter with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Dhanush444/Telugu-lora-adapter" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dhanush444/Telugu-lora-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Dhanush444/Telugu-lora-adapter" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dhanush444/Telugu-lora-adapter",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use Dhanush444/Telugu-lora-adapter with Docker Model Runner:
```
docker model run hf.co/Dhanush444/Telugu-lora-adapter
```

Model Card for Telugu-lora-adapter (Baasha)

Model Summary

Baasha (బాషా) is a LoRA adapter fine-tuned on top of Meta-Llama-3.1-8B-Instruct to make the base model respond only in natural, casual, spoken Telugu (వాడుక భాష) — never bookish/గ్రాంథిక Telugu, never Romanized ("Tenglish"), never English. It freely mixes common English loanwords inline within Telugu script, the way modern Telugu speakers actually talk.

This adapter is the conversational "brain" of a Telugu voice assistant pipeline, designed to sit between a Telugu speech-to-text model and a text-to-speech model.

Model Details

Model Description

Baasha is a QLoRA adapter trained via Unsloth on a custom-built, cleaned Telugu conversational dataset of 2,245 examples spanning everyday topics (greetings, identity, tech support, cricket, movies, food, festivals, study advice, emotional support, multi-turn dialogue, etc.). It was built specifically to correct the tendency of general-purpose LLMs to answer in English, Romanized Telugu, or overly formal/literary Telugu when prompted in Telugu.

Developed by: Dhanush444
Shared by: Dhanush444
Model type: LoRA adapter for causal language modeling (decoder-only transformer), conversational/instruction-tuned
Language(s) (NLP): Telugu (te) — casual/spoken register, with inline English loanwords
License: llama3.1 (inherits Meta Llama 3.1 Community License from the base model)
Finetuned from model: unsloth/Meta-Llama-3.1-8B-Instruct-unsloth-bnb-4bit

Model Sources

Repository: https://huggingface.co/Dhanush444/Telugu-lora-adapter
Related model (STT, same project): Dhanush444/whisper-telugu-medium-indicvoices — fine-tuned Whisper for Telugu speech-to-text, used upstream of this model in the full voice assistant pipeline.

Uses

Direct Use

Generating short, casual, spoken-Telugu conversational responses given a Telugu (or Telugu+English-mixed) user message and the Baasha system prompt. Suitable for chatbot/voice-assistant text generation where responses need to sound natural when converted to speech (2-3 sentence answers, no literary register).

Downstream Use

Designed to be used as the middle stage of a 3-part voice assistant pipeline:

User speech (Telugu) ↓ Whisper STT (Dhanush444/whisper-telugu-medium-indicvoices) ↓ Baasha LLM (this adapter, on Llama-3.1-8B-Instruct) ↓ Text-to-Speech (not yet included — pipeline in progress) ↓ Spoken reply to user

Intended end use case: a Telugu-speaking voice agent, eventually extended to handle real customer phone calls.

Out-of-Scope Use

Not intended for English-only conversation (will force casual Telugu register even when addressed in English).
Not intended for formal/literary/written Telugu (గ్రాంథిక) use cases — explicitly trained away from this.
Not evaluated for safety-critical, legal, medical, or financial advice use.
Not suited for long-form document generation — trained only for short (2-3 sentence) conversational replies.
Not evaluated for toxicity, bias, or harmful content filtering; no safety alignment beyond the base Llama 3.1 Instruct model was performed.

Bias, Risks, and Limitations

Compound questions: When a single user utterance contains two separate questions, the model tends to answer only one of them, ignoring the other. Partially mitigated via system prompt instruction, not fully fixed.
Meta-conversation recall: Questions referring back to "what did you just ask me" produce generic, evasive answers rather than genuine recall.
Multi-turn fact hallucination: In extended conversations, the model can misremember specific facts stated earlier (e.g., substituting one food item for another), while still correctly recalling other details (e.g., names). Likely caused by thin multi-turn training data (~30 of 2,245 examples).
Sparse-topic vagueness: Topics with fewer training examples (movies: 98, jokes: 102, tech problems: 106) produce noticeably vaguer answers than high-coverage topics (daily greetings: 197, identity: 198, simple math: 198).
Loss curve showed small drops at epoch boundaries (~step 281, ~562), suggesting mild memorization; not confirmed as a practical problem during inference testing, but noted for transparency.

Recommendations

Users should treat Baasha's multi-turn memory as unreliable for factual recall across more than 1-2 turns, and should not rely on it for compound or meta-conversational queries without further fine-tuning. Retraining with additional multi-turn and compound-question examples is recommended before production use.

How to Get Started with the Model

⚠️ Critical generation settings

These settings are required for correct output and were confirmed via direct A/B testing:

repetition_penalty must be 1.0. Any value above 1.0 (tested: 1.15, 1.3) breaks Telugu's agglutinative grammar and produces garbled non-words.
eos_token_id must explicitly include <|eot_id|> alongside the tokenizer's default EOS token. Omitting this causes mid-sentence/mid-word truncation at max_new_tokens instead of natural stopping.

System prompt

You are 'Baasha', a warm, kind, energetic AI voice assistant. Always reply in 100% casual, conversational Telugu SCRIPT (వాడుక భాష) — never Romanized Telugu, never English. The way modern, educated Telugu people actually speak daily. Freely mix common English WORDS for modern nouns/verbs (laptop, problem, help, idea, happy, restart, charge, battery) written inline within Telugu script sentences. If asked your name, always say your name is 'Baasha'. If the user's message contains multiple separate questions, answer EACH one briefly, in order, instead of only answering one. Keep answers short, crisp, friendly (2-3 sentences max).

Usage (Unsloth)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Dhanush444/Telugu-lora-adapter",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

eot_id = tokenizer.convert_tokens_to_ids("<|eot_id|>")

messages = [
    {"role": "system", "content": "<BAASHA_SYSTEM_PROMPT above>"},
    {"role": "user", "content": "నమస్తే, ఎలా ఉన్నావు?"},
]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to("cuda")

outputs = model.generate(
    inputs,
    max_new_tokens=150,
    eos_token_id=[tokenizer.eos_token_id, eot_id],
    repetition_penalty=1.0,
    temperature=0.7,
    do_sample=True,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Downloads last month: 29

Model tree for Dhanush444/Telugu-lora-adapter

Base model

meta-llama/Llama-3.1-8B

Finetuned

meta-llama/Llama-3.1-8B-Instruct

Quantized

unsloth/Meta-Llama-3.1-8B-Instruct-unsloth-bnb-4bit

Adapter

(21)

this model