File size: 1,081 Bytes
cc932d0 481cff9 cc932d0 481cff9 cc932d0 f8c43c9 cc932d0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
---
language:
- ms
---
# Noisy Translation Tiny T5
Trained on 1536 context length, able to translate malay, pasar malay (social media texts or local context), english, manglish, javanese, banjarese and indonesian to target language. It also able to maintain the text structure as it is and only translate necessary texts, eg, programming code.
Try it at https://huggingface.co/spaces/mesolitica/malaysian-translation
## how-to
```python
from transformers import T5ForConditionalGeneration, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
'mesolitica/translation-t5-tiny-standard-bahasa-cased',
use_fast=False
)
model = T5ForConditionalGeneration.from_pretrained(
'mesolitica/translation-t5-tiny-standard-bahasa-cased'
)
s = 'Hai, ada yang bisa saya bantu?'
input_ids = tokenizer.encode(f'terjemah ke Melayu: {s}', return_tensors = 'pt')
outputs = model.generate(input_ids, max_length = 100)
all_special_ids = [0, 1, 2]
outputs = [i for i in outputs[0] if i not in all_special_ids]
print(tokenizer.decode(outputs, spaces_between_special_tokens = False))
``` |