Papadopoulos, Dimitris commited on
Commit
a2f3ce5
1 Parent(s): 2357ded

Adding model weights, dicts and readme

Browse files
Files changed (7) hide show
  1. README.md +76 -0
  2. config.json +37 -0
  3. merges.txt +0 -0
  4. pytorch_model.bin +3 -0
  5. tokenizer_config.json +8 -0
  6. vocab-src.json +0 -0
  7. vocab-tgt.json +0 -0
README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - el
4
+ - en
5
+ tags:
6
+ - translation
7
+ widget:
8
+ - text: "Κάνω διδακτορικό στην υπολογιστική γλωσσολογία."
9
+ license: apache-2.0
10
+ metrics:
11
+ - bleu
12
+ ---
13
+
14
+ ## Greek to English NMT from Hellenic Army Academy (SSE) and Technical University of Crete (TUC)
15
+
16
+ * source languages: el
17
+ * target languages: en
18
+ * licence: apache-2.0
19
+ * dataset: Opus, CCmatrix
20
+ * model: transformer(fairseq)
21
+ * pre-processing: tokenization + BPE segmentation
22
+ * metrics: bleu, chrf
23
+
24
+ ### Model description
25
+
26
+ Trained using the Fairseq framework, transformer_iwslt_de_en architecture.\
27
+ BPE segmentation (20k codes).\
28
+ Mixed-case model.
29
+
30
+ ### How to use
31
+
32
+ ```
33
+ from transformers import FSMTTokenizer, FSMTForConditionalGeneration
34
+
35
+ mname = " <your_downloaded_model_folderpath_here> "
36
+
37
+ tokenizer = FSMTTokenizer.from_pretrained(mname)
38
+ model = FSMTForConditionalGeneration.from_pretrained(mname)
39
+
40
+ text = "Κάνω διδακτορικό στην υπολογιστική γλωσσολογία."
41
+
42
+ encoded = tokenizer.encode(text, return_tensors='pt')
43
+
44
+ outputs = model.generate(encoded, num_beams=5, num_return_sequences=5, early_stopping=True)
45
+ for i, output in enumerate(outputs):
46
+ i += 1
47
+ print(f"{i}: {output.tolist()}")
48
+
49
+ decoded = tokenizer.decode(output, skip_special_tokens=True)
50
+ print(f"{i}: {decoded}")
51
+ ```
52
+
53
+
54
+ ## Training data
55
+
56
+ Consolidated corpus from Opus and CC-Matrix (~6.6GB in total)
57
+
58
+
59
+ ## Eval results
60
+
61
+
62
+ Results on Tatoeba testset (EL-EN):
63
+
64
+ | BLEU | chrF |
65
+ | ------ | ------ |
66
+ | 79.3 | 0.795 |
67
+
68
+
69
+ Results on XNLI parallel (EL-EN):
70
+
71
+ | BLEU | chrF |
72
+ | ------ | ------ |
73
+ | 66.2 | 0.623 |
74
+
75
+ ### BibTeX entry and citation info
76
+ TODO
config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "FSMTForConditionalGeneration"
4
+ ],
5
+ "model_type": "fsmt",
6
+ "activation_dropout": 0.0,
7
+ "activation_function": "relu",
8
+ "attention_dropout": 0.0,
9
+ "d_model": 512,
10
+ "dropout": 0.3,
11
+ "init_std": 0.02,
12
+ "max_position_embeddings": 1024,
13
+ "num_hidden_layers": 6,
14
+ "src_vocab_size": 23224,
15
+ "tgt_vocab_size": 15288,
16
+ "langs": [
17
+ "el",
18
+ "en"
19
+ ],
20
+ "encoder_attention_heads": 4,
21
+ "encoder_ffn_dim": 1024,
22
+ "encoder_layerdrop": 0,
23
+ "encoder_layers": 6,
24
+ "decoder_attention_heads": 4,
25
+ "decoder_ffn_dim": 1024,
26
+ "decoder_layerdrop": 0,
27
+ "decoder_layers": 6,
28
+ "bos_token_id": 0,
29
+ "pad_token_id": 1,
30
+ "eos_token_id": 2,
31
+ "is_encoder_decoder": true,
32
+ "scale_embedding": true,
33
+ "tie_word_embeddings": false,
34
+ "num_beams": 5,
35
+ "early_stopping": false,
36
+ "length_penalty": 1.0
37
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cdc50bd4b28da36cd84c59cf3e6343add3c04f77b2d69dfd6ce39a4b2a87a01
3
+ size 205089118
tokenizer_config.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "langs": [
3
+ "el",
4
+ "en"
5
+ ],
6
+ "model_max_length": 1024,
7
+ "do_lower_case": false
8
+ }
vocab-src.json ADDED
The diff for this file is too large to render. See raw diff
 
vocab-tgt.json ADDED
The diff for this file is too large to render. See raw diff