huseinzol05
commited on
Commit
•
8a32724
1
Parent(s):
34f9a83
Update README.md
Browse files
README.md
CHANGED
@@ -6,18 +6,18 @@ language:
|
|
6 |
|
7 |
Trained on 1536 context length, able to translate malay, pasar malay (social media texts or local context), english, manglish, javanese, banjarese and indonesian to target language. It also able to maintain the text structure as it is and only translate necessary texts, eg, programming code.
|
8 |
|
9 |
-
Added more coding translation dataset, noisy b.cari.com.my translation
|
10 |
|
11 |
## how-to
|
12 |
|
13 |
```python
|
14 |
from transformers import T5ForConditionalGeneration, AutoTokenizer
|
15 |
tokenizer = AutoTokenizer.from_pretrained(
|
16 |
-
'mesolitica/t5-small-standard-bahasa-cased-v2',
|
17 |
use_fast=False
|
18 |
)
|
19 |
model = T5ForConditionalGeneration.from_pretrained(
|
20 |
-
'mesolitica/t5-small-standard-bahasa-cased-v2'
|
21 |
)
|
22 |
s = 'Hai, ada yang bisa saya bantu?'
|
23 |
input_ids = tokenizer.encode(f'terjemah ke Melayu: {s}', return_tensors = 'pt')
|
|
|
6 |
|
7 |
Trained on 1536 context length, able to translate malay, pasar malay (social media texts or local context), english, manglish, javanese, banjarese and indonesian to target language. It also able to maintain the text structure as it is and only translate necessary texts, eg, programming code.
|
8 |
|
9 |
+
Added more coding translation dataset, noisy b.cari.com.my translation, noisy ChatGPT4 translation and heavy postfilter.
|
10 |
|
11 |
## how-to
|
12 |
|
13 |
```python
|
14 |
from transformers import T5ForConditionalGeneration, AutoTokenizer
|
15 |
tokenizer = AutoTokenizer.from_pretrained(
|
16 |
+
'mesolitica/translation-t5-small-standard-bahasa-cased-v2',
|
17 |
use_fast=False
|
18 |
)
|
19 |
model = T5ForConditionalGeneration.from_pretrained(
|
20 |
+
'mesolitica/translation-t5-small-standard-bahasa-cased-v2'
|
21 |
)
|
22 |
s = 'Hai, ada yang bisa saya bantu?'
|
23 |
input_ids = tokenizer.encode(f'terjemah ke Melayu: {s}', return_tensors = 'pt')
|