File size: 2,400 Bytes
78aa4ee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
# English to Kamba
Author: Kathleen Siminyu
## Data
- The JW300 English-Kamba dataset, 58312 lines.
## Model
- Link to google drive folder with model(https://drive.google.com/open?id=1Y7Judrp0bakZh6mN5ScMOJ3pLr3knD2W)
## Analysis
- Tried out different BPE settings and managed some improvements on the baseline. Highest BLEU score I recorded was at BPE 25000. It might be worth exploring different bpe settings for the source and target languages. Results from different settings included in the analysis below.
BPE 4000
```sh
BLEU dev: 14.83
BLEU test: 24.96
```
BPE 5000
```sh
2020-05-10 17:08:17,436 - dev bleu: 15.00 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:08:53,304 - test bleu: 26.06 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
BPE 10000
```sh
2020-05-10 17:09:16,286 - dev bleu: 16.63 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:09:51,410 - test bleu: 27.20 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
BPE 15000
```sh
2020-05-10 17:10:16,787 - dev bleu: 16.72 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:10:53,812 - test bleu: 27.34 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
BPE 20000
```sh
2020-05-10 17:11:20,392 - dev bleu: 16.00 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:11:57,920 - test bleu: 27.04 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
BPE 30000
```sh
2020-05-10 17:13:45,951 - dev bleu: 14.93 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:14:28,275 - test bleu: 25.84 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
BPE 35000
```sh
2020-05-10 17:15:06,838 - dev bleu: 14.83 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:15:51,339 - test bleu: 25.19 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
BPE 40000
```sh
2020-05-10 17:16:32,342 - dev bleu: 15.03 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:17:19,359 - test bleu: 25.87 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
# Results
BPE 25000
```
2020-05-10 17:12:29,184 - dev bleu: 16.70 [Beam search decoding with beam size = 5 and alpha = 1.0]
2020-05-10 17:13:12,772 - test bleu: 27.90 [Beam search decoding with beam size = 5 and alpha = 1.0]
```
|