sappho192 commited on
Commit
330b53d
1 Parent(s): 0066a39

Add usage section

Browse files
Files changed (1) hide show
  1. README.md +39 -1
README.md CHANGED
@@ -16,4 +16,42 @@ tags:
16
 
17
  **FINAL FANTASY is a registered trademark of Square Enix Holdings Co., Ltd.**
18
 
19
- This project is detailed on the [Github repo](https://github.com/sappho192/ffxiv-ja-ko-translator).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
  **FINAL FANTASY is a registered trademark of Square Enix Holdings Co., Ltd.**
18
 
19
+ This project is detailed on the [Github repo](https://github.com/sappho192/ffxiv-ja-ko-translator).
20
+
21
+
22
+ # Usage
23
+
24
+ Refer to the `test_eval.ipynb`.
25
+
26
+ ## Inference
27
+
28
+ ```Python
29
+ from transformers import(
30
+ EncoderDecoderModel,
31
+ PreTrainedTokenizerFast,
32
+ BertJapaneseTokenizer,
33
+ )
34
+
35
+ import torch
36
+
37
+ encoder_model_name = "cl-tohoku/bert-base-japanese-v2"
38
+ decoder_model_name = "skt/kogpt2-base-v2"
39
+
40
+ src_tokenizer = BertJapaneseTokenizer.from_pretrained(encoder_model_name)
41
+ trg_tokenizer = PreTrainedTokenizerFast.from_pretrained(decoder_model_name)
42
+
43
+ # You should change following `./best_model` to the path of model **directory**
44
+ model = EncoderDecoderModel.from_pretrained("./best_model")
45
+
46
+ text = "ギルガメッシュ討伐戦"
47
+ # text = "ギルガメッシュ討伐戦に行ってきます。一緒に行きましょうか?"
48
+
49
+ def translate(text_src):
50
+ embeddings = src_tokenizer(text_src, return_attention_mask=False, return_token_type_ids=False, return_tensors='pt')
51
+ embeddings = {k: v for k, v in embeddings.items()}
52
+ output = model.generate(**embeddings)[0, 1:-1]
53
+ text_trg = trg_tokenizer.decode(output.cpu())
54
+ return text_trg
55
+
56
+ print(translate(text))
57
+ ```