|
--- |
|
license: apache-2.0 |
|
language: |
|
- bn |
|
- en |
|
library_name: transformers |
|
tags: |
|
- BanglaT5 |
|
- Transformers |
|
- Transliteration |
|
--- |
|
# BanglishToBanglaTransliteration |
|
|
|
**Model Description:** |
|
|
|
The `BanglishToBanglaTransliteration` model is designed to transliterate text written in Banglish (Bengali language written using the Latin script) into proper Bengali script. This model is useful for processing and converting text that has been transliterated into Latin characters, which is common in digital communication among Bengali speakers. |
|
|
|
## Model Details |
|
|
|
**Model Type:** Transformer-based Encoder-Decoder Model |
|
**Languages:** Bengali (Bangla) |
|
**Training Data:** The model was trained on a dataset of Banglish and corresponding Bengali sentences. |
|
|
|
## Usage |
|
|
|
### Installation |
|
|
|
To use this model, you need to install the `transformers` library by Hugging Face: |
|
|
|
```bash |
|
pip install transformers |
|
``` |
|
|
|
### How to Use |
|
|
|
Here is an example of how to use the `BanglishToBanglaTransliteration` model with the `transformers` library: |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("kazalbrur/BanglishToBanglaTransliteration") |
|
model = AutoModelForSeq2SeqLM.from_pretrained("kazalbrur/BanglishToBanglaTransliteration") |
|
|
|
banglish_text = "apni kemon achen?" |
|
inputs = tokenizer.encode(banglish_text, return_tensors="pt") |
|
outputs = model.generate(inputs) |
|
bangla_text = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
|
print(bangla_text) |
|
``` |
|
|
|
### Training Details |
|
|
|
**Hyperparameters:** |
|
|
|
- **Embedding Dimension:** 256 |
|
- **Fully Connected Dimension:** 512 |
|
- **Number of Layers:** 4 |
|
- **Number of Attention Heads:** 8 |
|
- **Dropout Rate:** 0.1 |
|
- **Epochs:** 200 |
|
- **Batch Size:** 256 |
|
|
|
**Optimizer:** Adam with a custom learning rate schedule. |
|
|
|
### Training Procedure |
|
|
|
The model was trained using a Transformer architecture, incorporating positional encodings, multi-head attention mechanisms, and feed-forward neural networks. It was optimized using a custom learning rate schedule and Adam optimizer. |
|
|
|
**Loss Function:** Sparse Categorical Crossentropy, masked to ignore padding tokens. |
|
|
|
**Accuracy Function:** Calculated based on the exact match of tokens, excluding padding tokens. |
|
|
|
### Evaluation |
|
|
|
The model's performance was evaluated using standard NLP metrics. The accuracy and loss during training were tracked and can be plotted to visualize model performance. |
|
|
|
## Limitations |
|
|
|
The model may not perform well on out-of-vocabulary words or phrases that were not included in the training data. Additionally, transliteration errors can occur if the input Banglish text is not phonetically accurate. |
|
|
|
## Future Work |
|
|
|
Future improvements may include expanding the training dataset, fine-tuning on more diverse datasets, and integrating with broader NLP tasks. |
|
|
|
## Citation |
|
|
|
If you use this model, please cite it as follows: |
|
|
|
``` |
|
@misc{banglishtobanglatransliteration, |
|
author = {Kazal Chandra Barman}, |
|
title = {BanglishToBanglaTransliteration}, |
|
year = {2024}, |
|
url = {https://huggingface.co/kazalbrur/BanglishToBanglaTransliteration} |
|
} |
|
``` |
|
|
|
## Acknowledgements |
|
|
|
Special thanks to [https://www.kaggle.com/datasets/kazalnext/banglish-to-bangla-dataset] for providing the dataset and [csebuetnlp/banglat5] for facilitating the development of this model. |
|
|
|
--- |
