|
--- |
|
library_name: transformers |
|
license: apache-2.0 |
|
datasets: |
|
- cfilt/iitb-english-hindi |
|
language: |
|
- en |
|
- hi |
|
pipeline_tag: translation |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
|
|
|
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
This model is a fine-tuned version of the GEMMA 2B multilingual transformer, specifically optimized for translating text from English to Hindi. It leverages the capabilities of the original GEMMA architecture to provide accurate and efficient translations. |
|
|
|
-Model Name: Gemma-2b-mt-Hindi-Fintuned |
|
-Model Type: Language Translation Model |
|
-Base Model: Gemma-2b |
|
-Task: English to Hindi Translation |
|
-Framework: Transformers |
|
|
|
### Model Sources [optional] |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
This model can be directly used for translating English text to Hindi. It is suitable for various applications such as: |
|
-Localization of content |
|
-Cross-lingual communication |
|
-Educational tools for language learning |
|
-Multilingual content creation |
|
|
|
### Downstream Use [optional] |
|
|
|
The model can be integrated into larger systems or applications that require English to Hindi translation capabilities, such as: |
|
-Machine translation services |
|
-Multilingual chatbots |
|
-Content management systems for multilingual websites |
|
|
|
[More Information Needed] |
|
|
|
### Out-of-Scope Use |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
-The model may struggle with idiomatic expressions or culturally specific content. |
|
-There might be potential biases in the training data that could affect translation quality. |
|
-The model's performance on specialized or technical content may vary. |
|
-It may have limitations in handling complex grammatical structures or maintaining context in longer texts. |
|
|
|
### Recommendations |
|
|
|
-It's recommended to use the model in conjunction with human translators for high-stakes or nuanced translations. |
|
-Regular evaluation and fine-tuning with diverse and representative data can help mitigate biases and improve performance. |
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model: |
|
|
|
---------------------------------------------------------------------------------------- |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
#Load the model and tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("Satwik11/gemma-2b-mt-Hindi-Fintuned") |
|
model = AutoModelForCausalLM.from_pretrained("Satwik11/gemma-2b-mt-Hindi-Fintuned") |
|
|
|
def generate_translation(prompt, max_length=90): |
|
# Prepare the input |
|
inputs = tokenizer(prompt, return_tensors='pt') |
|
|
|
# Generate the translation |
|
outputs = model.generate(**inputs, max_length=max_length) |
|
|
|
# Decode the generated output |
|
translated_text = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
|
return translated_text |
|
|
|
#Test the model with some example sentences |
|
test_sentences = [ |
|
"Today is August 19.The maximum temperature is 70 degrees Fahrenheit" |
|
] |
|
|
|
for sentence in test_sentences: |
|
prompt = f"Translate the following English text to Hindi: {sentence}" |
|
translation = generate_translation(prompt) |
|
print(translation) |
|
|
|
----------------------------------------------------------------------------------------------- |
|
## Training Details |
|
|
|
### Training Data |
|
|
|
The model was fine-tuned on the cfilt/iitb-english-hindi dataset, which contains English-Hindi sentence pairs. For more details about the dataset, refer to the dataset card on Hugging Face. |
|
|
|
|
|
|
|
## Model Card Contact |
|
|
|
For more information, please contact the model creators through the Hugging Face model repository: https://www.linkedin.com/in/satwik-sinha/ |
