--- language: - vi - lo tags: - translation license: mit widget: - text: "ຂ້ອຍຢາກຊື້ປຶ້ມ" inference: parameters: max_length: 200 pipeline_tag: translation library_name: transformers --- # Lao - Vietnamese Translation Model In the domain of natural language processing (NLP), the development of translation models tailored for low-resource languages represents a critical endeavor to facilitate cross-cultural communication and knowledge exchange. In response to this challenge, we present a novel and impactful contribution: a translation model specifically designed to bridge the linguistic gap between Lao and Vietnamese. Lao, a language spoken primarily in Laos and parts of Thailand, presents inherent challenges for machine translation due to its low-resource nature, characterized by limited parallel corpora and linguistic resources. Vietnamese, a language spoken by millions worldwide, shares some linguistic similarities with Lao, making it an ideal target language for translation purposes. Leveraging the power of the Transformer-based T5 model, we have developed a robust translation system for the Lao-Vietnamese language pair. The T5 model, renowned for its versatility and effectiveness across various NLP tasks, serves as the cornerstone of our approach. Through fine-tuning on a curated dataset of Lao-Vietnamese parallel texts, we have endeavored to enhance translation accuracy and fluency, thus enabling smoother communication between speakers of these languages. Our work represents a significant advancement in the field of machine translation, particularly for low-resource languages like Lao. By harnessing state-of-the-art NLP techniques and focusing on the specific linguistic nuances of the Lao-Vietnamese language pair, we aim to provide a valuable resource for facilitating cross-linguistic communication and cultural exchange. ## How to use ### On GPU ```python from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("minhtoan/t5-translate-lao-vietnamese") model = AutoModelForSeq2SeqLM.from_pretrained("minhtoan/t5-translate-lao-vietnamese") model.cuda() src = "ຂ້ອຍຢາກຊື້ປຶ້ມາ" tokenized_text = tokenizer.encode(src, return_tensors="pt").cuda() model.eval() translate_ids = model.generate(tokenized_text, max_length=200) output = tokenizer.decode(translate_ids[0], skip_special_tokens=True) output ``` 'Tôi muốn mua một cuốn sách' ### On CPU ```python from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("minhtoan/t5-translate-lao-vietnamese") model = AutoModelForSeq2SeqLM.from_pretrained("minhtoan/t5-translate-lao-vietnamese") src = "ຂ້ອຍຢາກຊື້ປຶ້ມ" input_ids = tokenizer(src, max_length=200, return_tensors="pt", padding="max_length", truncation=True).input_ids outputs = model.generate(input_ids=input_ids, max_new_tokens=200) output = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0] output ``` 'Tôi muốn mua một cuốn sách' ## Author ` Phan Minh Toan `