revanthkumar1999/gemma-2-Indian_languages-to-eng

Model Card for Gemma 2 Multi-Task Translation Model

Model Details

Model Description

This model is a multi-task fine-tuned version of google/gemma-2-2b for machine translation between English and four target languages:

• English → Telugu
• English → Tamil
• English → Hindi
• English → Nepali

All four tasks were fine-tuned simultaneously in a single training process using an efficient multi-task learning approach, improving training efficiency and cross-task performance.
The model achieves state-of-the-art scores across evaluation datasets.

• Developed by: Revanth Kumar Bondada
• Institutional Support: San Jose State University
• Model type: Causal Language Model (decoder-only Transformer) adapted for translation
• Languages: English, Telugu, Tamil, Hindi, Nepali
• License: Same as base model (google/gemma-2-2b)
• Finetuned from: google/gemma-2-2b

Model Sources

• Repository: GitHub Link
• Hugging Face Model: Model Link

Uses

Direct Use

• Translate English text into Telugu, Tamil, Hindi, or Nepali.
• Educational, research, and multilingual NLP applications.

Downstream Use

• Integration into multilingual chatbots, educational apps, and content localization pipelines.
• Further fine-tuning for domain-specific translation needs.

Out-of-Scope Use

• Translation for languages outside the trained set.
• Use in sensitive or high-risk contexts without human verification.

Bias, Risks, and Limitations

• Performance may be lower for domain-specific jargon.
• Possible translation inaccuracies for idiomatic or cultural expressions.

Recommendations

• Always perform human validation for critical translation tasks.
• Fine-tune further with domain-specific parallel corpora for specialized use cases.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "revanthkumar1999/gemma-2-Indian_languages-to-eng"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

prompt = "Translate to Telugu: The weather is pleasant today."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

• Multi-parallel datasets for English ↔ Telugu, English ↔ Tamil, English ↔ Hindi, and English ↔ Nepali.
• Balanced sampling to ensure equitable performance across all tasks.

Training Procedure

• Method: Multi-task fine-tuning
• Precision: Mixed precision (fp16)
• Optimizer: AdamW
• Learning rate: 2e-5
• Batch size: 32
• Epochs: 4
• Context length: 4096 tokens

Compute Infrastructure

• Hardware: NVIDIA RTX 4090 GPU (24GB VRAM)
• Framework: PyTorch + PEFT + Transformers
• Environment: Local workstation with CUDA 12.1

Evaluation

Testing Data

• Held-out test sets from the same sources as training data.

Metrics

Metric	Score
BLEU	87%
METEOR	91%
ROUGE	90%

Results Summary

The model demonstrated strong performance across all four language pairs, with balanced accuracy and fluency, benefiting from the shared multilingual fine-tuning process.

Environmental Impact

• Hardware Type: NVIDIA RTX 4090
• Training Time: ~10 hours
• Location: [ San Jose, CA]
• Carbon Emitted: ~5.3 kg CO₂eq (estimated)

Technical Specifications

Model Architecture

• 2B parameter Gemma 2 decoder-only Transformer
• Adapted with multi-task training for translation

Framework Versions

• Transformers 4.43.0
• PEFT 0.15.2.dev0
• PyTorch 2.3.0