README.md

---
license: mit
datasets:
- cmu_hinglish_dog
language:
- en
library_name: transformers
pipeline_tag: translation
tags:
- hinglish
- translation
- hinglish to english
- language translation
- keras
- keras nlp
- nlp
- transformers
- gemma2b
---
# Project Hinglish - A Hinglish to English Language Translater.

Project Hinglish aims to develop a high-performance language translation model capable of translating Hinglish (a blend of Hindi and English commonly used in informal communication in India) to standard English.
The model is fine-tuned over gemma-2b using PEFT(LoRA) method using the rank 128. Aim of this model is for handling the unique syntactical and lexical characteristics of Hinglish.

# Fine-Tune Method:

- **Fine-Tuning Approach Using PEFT (LoRA):** The fine-tuning employs Parameter-efficient Fine Tuning (PEFT) techniques, particularly using LoRA (Low-Rank Adaptation). LoRA modifies a pre-trained model efficiently by introducing low-rank matrices that adapt the model’s attention and feed-forward layers. This method allows significant model adaptation with minimal updates to the parameters, preserving the original model's strengths while adapting it effectively to the nuances of Hinglish.
- **Dataset:** cmu_hinglish_dog + Combination of sentences taken from my own dialy life chats with friends and Uber Messages.

# Example Output

![Example IO](io1.png)

# Usage
``` python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rudrashah/RLM-hinglish-translator")
model = AutoModelForCausalLM.from_pretrained("rudrashah/RLM-hinglish-translator")

template = "Hinglish:\n{hi_en}\n\nEnglish:\n{en}" #THIS IS MOST IMPORTANT, WITHOUT THIS IT WILL GIVE RANDOM OUTPUT
input_text = tokenizer(template.format(hi_en="aapka name kya hai?",en=""),return_tensors="pt")

output = model.generate(**input_text)
print(tokenizer.decode(output[0]))
```