--- language: - af - de - en - fy - gos - hrx - lb - multilingual - nds - nl tags: - translation - opus-mt-tc license: cc-by-4.0 model-index: - name: opus-mt-tc-big-gmw-gmw results: - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: news-test2008 type: news-test2008 args: deu-eng metrics: - name: BLEU type: bleu value: 27.2 - name: chr-F type: chrf value: 0.54584 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: news-test2008 type: news-test2008 args: eng-deu metrics: - name: BLEU type: bleu value: 23.7 - name: chr-F type: chrf value: 0.53204 - task: name: Translation afr-deu type: translation args: afr-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: afr-deu metrics: - name: BLEU type: bleu value: 50.3 - name: chr-F type: chrf value: 0.68633 - task: name: Translation afr-eng type: translation args: afr-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: afr-eng metrics: - name: BLEU type: bleu value: 56.4 - name: chr-F type: chrf value: 0.70502 - task: name: Translation afr-nld type: translation args: afr-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: afr-nld metrics: - name: BLEU type: bleu value: 55.5 - name: chr-F type: chrf value: 0.71500 - task: name: Translation bar-bar type: translation args: bar-bar dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: bar-bar metrics: - name: BLEU type: bleu value: 54.1 - name: chr-F type: chrf value: 0.57238 - task: name: Translation deu-afr type: translation args: deu-afr dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: deu-afr metrics: - name: BLEU type: bleu value: 54.2 - name: chr-F type: chrf value: 0.70191 - task: name: Translation deu-deu type: translation args: deu-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: deu-deu metrics: - name: BLEU type: bleu value: 34.6 - name: chr-F type: chrf value: 0.57304 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: deu-eng metrics: - name: BLEU type: bleu value: 48.4 - name: chr-F type: chrf value: 0.65919 - task: name: Translation deu-nds type: translation args: deu-nds dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: deu-nds metrics: - name: BLEU type: bleu value: 23.2 - name: chr-F type: chrf value: 0.48028 - task: name: Translation deu-nld type: translation args: deu-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: deu-nld metrics: - name: BLEU type: bleu value: 54.4 - name: chr-F type: chrf value: 0.71366 - task: name: Translation deu-tpi type: translation args: deu-tpi dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: deu-tpi metrics: - name: BLEU type: bleu value: 42.9 - name: chr-F type: chrf value: 0.61690 - task: name: Translation drt-deu type: translation args: drt-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: drt-deu metrics: - name: BLEU type: bleu value: 53.4 - name: chr-F type: chrf value: 0.71148 - task: name: Translation drt-eng type: translation args: drt-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: drt-eng metrics: - name: BLEU type: bleu value: 41.9 - name: chr-F type: chrf value: 0.60142 - task: name: Translation drt-fry type: translation args: drt-fry dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: drt-fry metrics: - name: BLEU type: bleu value: 26.3 - name: chr-F type: chrf value: 0.43513 - task: name: Translation drt-nld type: translation args: drt-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: drt-nld metrics: - name: BLEU type: bleu value: 62.9 - name: chr-F type: chrf value: 0.73241 - task: name: Translation eng-afr type: translation args: eng-afr dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: eng-afr metrics: - name: BLEU type: bleu value: 56.4 - name: chr-F type: chrf value: 0.71940 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: eng-deu metrics: - name: BLEU type: bleu value: 41.8 - name: chr-F type: chrf value: 0.62912 - task: name: Translation eng-eng type: translation args: eng-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: eng-eng metrics: - name: BLEU type: bleu value: 66.3 - name: chr-F type: chrf value: 0.80136 - task: name: Translation eng-nld type: translation args: eng-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: eng-nld metrics: - name: BLEU type: bleu value: 54.3 - name: chr-F type: chrf value: 0.70929 - task: name: Translation eng-sco type: translation args: eng-sco dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: eng-sco metrics: - name: BLEU type: bleu value: 21.4 - name: chr-F type: chrf value: 0.50135 - task: name: Translation eng-srn type: translation args: eng-srn dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: eng-srn metrics: - name: BLEU type: bleu value: 37.0 - name: chr-F type: chrf value: 0.65920 - task: name: Translation eng-tpi type: translation args: eng-tpi dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: eng-tpi metrics: - name: BLEU type: bleu value: 34.5 - name: chr-F type: chrf value: 0.53886 - task: name: Translation enm-deu type: translation args: enm-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: enm-deu metrics: - name: BLEU type: bleu value: 36.4 - name: chr-F type: chrf value: 0.52972 - task: name: Translation enm-eng type: translation args: enm-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: enm-eng metrics: - name: BLEU type: bleu value: 22.1 - name: chr-F type: chrf value: 0.44876 - task: name: Translation enm-fry type: translation args: enm-fry dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: enm-fry metrics: - name: BLEU type: bleu value: 58.7 - name: chr-F type: chrf value: 0.63860 - task: name: Translation enm-ltz type: translation args: enm-ltz dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: enm-ltz metrics: - name: BLEU type: bleu value: 20.1 - name: chr-F type: chrf value: 0.49027 - task: name: Translation enm-nld type: translation args: enm-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: enm-nld metrics: - name: BLEU type: bleu value: 33.0 - name: chr-F type: chrf value: 0.53731 - task: name: Translation fry-deu type: translation args: fry-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: fry-deu metrics: - name: BLEU type: bleu value: 28.7 - name: chr-F type: chrf value: 0.48820 - task: name: Translation fry-eng type: translation args: fry-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: fry-eng metrics: - name: BLEU type: bleu value: 24.5 - name: chr-F type: chrf value: 0.40304 - task: name: Translation fry-ltz type: translation args: fry-ltz dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: fry-ltz metrics: - name: BLEU type: bleu value: 22.2 - name: chr-F type: chrf value: 0.41580 - task: name: Translation fry-nld type: translation args: fry-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: fry-nld metrics: - name: BLEU type: bleu value: 40.5 - name: chr-F type: chrf value: 0.54939 - task: name: Translation gos-afr type: translation args: gos-afr dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: gos-afr metrics: - name: BLEU type: bleu value: 43.2 - name: chr-F type: chrf value: 0.59703 - task: name: Translation gos-deu type: translation args: gos-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: gos-deu metrics: - name: BLEU type: bleu value: 25.4 - name: chr-F type: chrf value: 0.45302 - task: name: Translation gos-eng type: translation args: gos-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: gos-eng metrics: - name: BLEU type: bleu value: 23.9 - name: chr-F type: chrf value: 0.37587 - task: name: Translation gos-fry type: translation args: gos-fry dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: gos-fry metrics: - name: BLEU type: bleu value: 22.2 - name: chr-F type: chrf value: 0.44738 - task: name: Translation gos-nld type: translation args: gos-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: gos-nld metrics: - name: BLEU type: bleu value: 26.1 - name: chr-F type: chrf value: 0.45701 - task: name: Translation gsw-deu type: translation args: gsw-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: gsw-deu metrics: - name: BLEU type: bleu value: 27.5 - name: chr-F type: chrf value: 0.44989 - task: name: Translation gsw-nld type: translation args: gsw-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: gsw-nld metrics: - name: BLEU type: bleu value: 23.6 - name: chr-F type: chrf value: 0.43814 - task: name: Translation hrx-deu type: translation args: hrx-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: hrx-deu metrics: - name: BLEU type: bleu value: 30.0 - name: chr-F type: chrf value: 0.51840 - task: name: Translation hrx-eng type: translation args: hrx-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: hrx-eng metrics: - name: BLEU type: bleu value: 29.2 - name: chr-F type: chrf value: 0.42778 - task: name: Translation lim-deu type: translation args: lim-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: lim-deu metrics: - name: BLEU type: bleu value: 58.6 - name: chr-F type: chrf value: 0.68147 - task: name: Translation lim-eng type: translation args: lim-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: lim-eng metrics: - name: BLEU type: bleu value: 59.7 - name: chr-F type: chrf value: 0.66629 - task: name: Translation lim-nld type: translation args: lim-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: lim-nld metrics: - name: BLEU type: bleu value: 47.5 - name: chr-F type: chrf value: 0.68145 - task: name: Translation ltz-deu type: translation args: ltz-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: ltz-deu metrics: - name: BLEU type: bleu value: 21.0 - name: chr-F type: chrf value: 0.37005 - task: name: Translation ltz-eng type: translation args: ltz-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: ltz-eng metrics: - name: BLEU type: bleu value: 30.1 - name: chr-F type: chrf value: 0.37764 - task: name: Translation ltz-fry type: translation args: ltz-fry dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: ltz-fry metrics: - name: BLEU type: bleu value: 27.3 - name: chr-F type: chrf value: 0.47498 - task: name: Translation ltz-nld type: translation args: ltz-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: ltz-nld metrics: - name: BLEU type: bleu value: 26.4 - name: chr-F type: chrf value: 0.32392 - task: name: Translation multi-multi type: translation args: multi-multi dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: multi-multi metrics: - name: BLEU type: bleu value: 40.4 - name: chr-F type: chrf value: 0.59400 - task: name: Translation nds-deu type: translation args: nds-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nds-deu metrics: - name: BLEU type: bleu value: 45.5 - name: chr-F type: chrf value: 0.63898 - task: name: Translation nds-eng type: translation args: nds-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nds-eng metrics: - name: BLEU type: bleu value: 38.4 - name: chr-F type: chrf value: 0.55112 - task: name: Translation nds-nld type: translation args: nds-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nds-nld metrics: - name: BLEU type: bleu value: 49.8 - name: chr-F type: chrf value: 0.66676 - task: name: Translation nld-afr type: translation args: nld-afr dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nld-afr metrics: - name: BLEU type: bleu value: 62.3 - name: chr-F type: chrf value: 0.76610 - task: name: Translation nld-deu type: translation args: nld-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nld-deu metrics: - name: BLEU type: bleu value: 56.7 - name: chr-F type: chrf value: 0.73047 - task: name: Translation nld-eng type: translation args: nld-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nld-eng metrics: - name: BLEU type: bleu value: 60.2 - name: chr-F type: chrf value: 0.73940 - task: name: Translation nld-fry type: translation args: nld-fry dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nld-fry metrics: - name: BLEU type: bleu value: 31.0 - name: chr-F type: chrf value: 0.47959 - task: name: Translation nld-nds type: translation args: nld-nds dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nld-nds metrics: - name: BLEU type: bleu value: 20.0 - name: chr-F type: chrf value: 0.43743 - task: name: Translation nld-nld type: translation args: nld-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nld-nld metrics: - name: BLEU type: bleu value: 44.9 - name: chr-F type: chrf value: 0.63646 - task: name: Translation nld-sco type: translation args: nld-sco dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: nld-sco metrics: - name: BLEU type: bleu value: 37.0 - name: chr-F type: chrf value: 0.68223 - task: name: Translation ofs-bar type: translation args: ofs-bar dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: ofs-bar metrics: - name: BLEU type: bleu value: 22.3 - name: chr-F type: chrf value: 0.31888 - task: name: Translation pdc-deu type: translation args: pdc-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: pdc-deu metrics: - name: BLEU type: bleu value: 39.5 - name: chr-F type: chrf value: 0.49683 - task: name: Translation pdc-eng type: translation args: pdc-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: pdc-eng metrics: - name: BLEU type: bleu value: 34.6 - name: chr-F type: chrf value: 0.47919 - task: name: Translation sco-eng type: translation args: sco-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: sco-eng metrics: - name: BLEU type: bleu value: 25.9 - name: chr-F type: chrf value: 0.43718 - task: name: Translation sco-nld type: translation args: sco-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: sco-nld metrics: - name: BLEU type: bleu value: 43.7 - name: chr-F type: chrf value: 0.55897 - task: name: Translation srn-eng type: translation args: srn-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: srn-eng metrics: - name: BLEU type: bleu value: 42.2 - name: chr-F type: chrf value: 0.43374 - task: name: Translation stq-deu type: translation args: stq-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: stq-deu metrics: - name: BLEU type: bleu value: 35.5 - name: chr-F type: chrf value: 0.56220 - task: name: Translation stq-eng type: translation args: stq-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: stq-eng metrics: - name: BLEU type: bleu value: 28.9 - name: chr-F type: chrf value: 0.43818 - task: name: Translation stq-nld type: translation args: stq-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: stq-nld metrics: - name: BLEU type: bleu value: 27.0 - name: chr-F type: chrf value: 0.49046 - task: name: Translation swg-eng type: translation args: swg-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: swg-eng metrics: - name: BLEU type: bleu value: 22.2 - name: chr-F type: chrf value: 0.40521 - task: name: Translation swg-nld type: translation args: swg-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: swg-nld metrics: - name: BLEU type: bleu value: 32.8 - name: chr-F type: chrf value: 0.38592 - task: name: Translation tpi-deu type: translation args: tpi-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: tpi-deu metrics: - name: BLEU type: bleu value: 26.8 - name: chr-F type: chrf value: 0.51890 - task: name: Translation tpi-eng type: translation args: tpi-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: tpi-eng metrics: - name: BLEU type: bleu value: 27.1 - name: chr-F type: chrf value: 0.43439 - task: name: Translation zea-deu type: translation args: zea-deu dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: zea-deu metrics: - name: BLEU type: bleu value: 60.4 - name: chr-F type: chrf value: 0.72725 - task: name: Translation zea-eng type: translation args: zea-eng dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: zea-eng metrics: - name: BLEU type: bleu value: 50.7 - name: chr-F type: chrf value: 0.56884 - task: name: Translation zea-fry type: translation args: zea-fry dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: zea-fry metrics: - name: BLEU type: bleu value: 28.6 - name: chr-F type: chrf value: 0.44983 - task: name: Translation zea-nds type: translation args: zea-nds dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: zea-nds metrics: - name: BLEU type: bleu value: 34.6 - name: chr-F type: chrf value: 0.51589 - task: name: Translation zea-nld type: translation args: zea-nld dataset: name: tatoeba-test-v2020-07-28-v2021-08-07 type: tatoeba_mt args: zea-nld metrics: - name: BLEU type: bleu value: 47.5 - name: chr-F type: chrf value: 0.71248 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2009 type: wmt-2009-news args: deu-eng metrics: - name: BLEU type: bleu value: 25.9 - name: chr-F type: chrf value: 0.53749 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2009 type: wmt-2009-news args: eng-deu metrics: - name: BLEU type: bleu value: 22.9 - name: chr-F type: chrf value: 0.53283 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2010 type: wmt-2010-news args: deu-eng metrics: - name: BLEU type: bleu value: 30.6 - name: chr-F type: chrf value: 0.58356 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2010 type: wmt-2010-news args: eng-deu metrics: - name: BLEU type: bleu value: 25.8 - name: chr-F type: chrf value: 0.54886 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2011 type: wmt-2011-news args: deu-eng metrics: - name: BLEU type: bleu value: 26.3 - name: chr-F type: chrf value: 0.54883 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2011 type: wmt-2011-news args: eng-deu metrics: - name: BLEU type: bleu value: 23.1 - name: chr-F type: chrf value: 0.52712 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2012 type: wmt-2012-news args: deu-eng metrics: - name: BLEU type: bleu value: 28.5 - name: chr-F type: chrf value: 0.56160 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2012 type: wmt-2012-news args: eng-deu metrics: - name: BLEU type: bleu value: 23.3 - name: chr-F type: chrf value: 0.52662 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2013 type: wmt-2013-news args: deu-eng metrics: - name: BLEU type: bleu value: 31.4 - name: chr-F type: chrf value: 0.57770 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2013 type: wmt-2013-news args: eng-deu metrics: - name: BLEU type: bleu value: 27.8 - name: chr-F type: chrf value: 0.55774 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2014-deen type: wmt-2014-news args: deu-eng metrics: - name: BLEU type: bleu value: 33.2 - name: chr-F type: chrf value: 0.59826 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2014-deen type: wmt-2014-news args: eng-deu metrics: - name: BLEU type: bleu value: 29.6 - name: chr-F type: chrf value: 0.59441 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2015-ende type: wmt-2015-news args: deu-eng metrics: - name: BLEU type: bleu value: 33.4 - name: chr-F type: chrf value: 0.59660 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2015-ende type: wmt-2015-news args: eng-deu metrics: - name: BLEU type: bleu value: 32.3 - name: chr-F type: chrf value: 0.59889 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2016-ende type: wmt-2016-news args: deu-eng metrics: - name: BLEU type: bleu value: 39.8 - name: chr-F type: chrf value: 0.64736 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2016-ende type: wmt-2016-news args: eng-deu metrics: - name: BLEU type: bleu value: 38.3 - name: chr-F type: chrf value: 0.64429 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2017-ende type: wmt-2017-news args: deu-eng metrics: - name: BLEU type: bleu value: 35.2 - name: chr-F type: chrf value: 0.60933 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2017-ende type: wmt-2017-news args: eng-deu metrics: - name: BLEU type: bleu value: 30.7 - name: chr-F type: chrf value: 0.59258 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2018-ende type: wmt-2018-news args: deu-eng metrics: - name: BLEU type: bleu value: 42.6 - name: chr-F type: chrf value: 0.66796 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2018-ende type: wmt-2018-news args: eng-deu metrics: - name: BLEU type: bleu value: 46.5 - name: chr-F type: chrf value: 0.69605 - task: name: Translation deu-eng type: translation args: deu-eng dataset: name: newstest2019-deen type: wmt-2019-news args: deu-eng metrics: - name: BLEU type: bleu value: 39.8 - name: chr-F type: chrf value: 0.63766 - task: name: Translation eng-deu type: translation args: eng-deu dataset: name: newstest2019-ende type: wmt-2019-news args: eng-deu metrics: - name: BLEU type: bleu value: 43.3 - name: chr-F type: chrf value: 0.66880 --- # opus-mt-tc-big-gmw-gmw ## Table of Contents - [Model Details](#model-details) - [Uses](#uses) - [Risks, Limitations and Biases](#risks-limitations-and-biases) - [How to Get Started With the Model](#how-to-get-started-with-the-model) - [Training](#training) - [Evaluation](#evaluation) - [Citation Information](#citation-information) - [Acknowledgements](#acknowledgements) ## Model Details Neural machine translation model for translating from West Germanic languages (gmw) to West Germanic languages (gmw). This model is part of the [OPUS-MT project](https://github.com/Helsinki-NLP/Opus-MT), an effort to make neural machine translation models widely available and accessible for many languages in the world. All models are originally trained using the amazing framework of [Marian NMT](https://marian-nmt.github.io/), an efficient NMT implementation written in pure C++. The models have been converted to pyTorch using the transformers library by huggingface. Training data is taken from [OPUS](https://opus.nlpl.eu/) and training pipelines use the procedures of [OPUS-MT-train](https://github.com/Helsinki-NLP/Opus-MT-train). **Model Description:** - **Developed by:** Language Technology Research Group at the University of Helsinki - **Model Type:** Translation (transformer-big) - **Release**: 2022-08-11 - **License:** CC-BY-4.0 - **Language(s):** - Source Language(s): afr deu eng enm fry gos gsw hrx ksh ltz multi nds nld pdc sco stq swg tpi yid - Target Language(s): afr ang deu eng enm fry gos ltz multi nds nld sco tpi yid - Language Pair(s): afr-deu afr-eng afr-nld deu-afr deu-deu deu-eng deu-nds deu-nld eng-afr eng-deu eng-eng eng-nld fry-eng fry-nld gos-deu gos-eng gos-nld hrx-deu hrx-eng ltz-deu ltz-eng ltz-nld multi-multi nds-deu nds-eng nds-nld nld-afr nld-deu nld-eng nld-fry nld-nds nld-nld - Valid Target Language Labels: >>act<< >>afr<< >>afs<< >>aig<< >>ang<< >>ang_Latn<< >>bah<< >>bar<< >>bis<< >>bjs<< >>brc<< >>bzj<< >>bzj_Latn<< >>bzk<< >>cim<< >>dcr<< >>deu<< >>djk<< >>djk_Latn<< >>drt<< >>drt_Latn<< >>dum<< >>eng<< >>enm<< >>enm_Latn<< >>fpe<< >>frk<< >>frr<< >>fry<< >>gcl<< >>gct<< >>geh<< >>gmh<< >>gml<< >>goh<< >>gos<< >>gpe<< >>gsw<< >>gul<< >>gyn<< >>hrx<< >>hrx_Latn<< >>hwc<< >>icr<< >>jam<< >>jvd<< >>kri<< >>ksh<< >>kww<< >>lim<< >>lng<< >>ltz<< >>mhn<< >>nds<< >>nld<< >>odt<< >>ofs<< >>ofs_Latn<< >>oor<< >>osx<< >>pcm<< >>pdc<< >>pdt<< >>pey<< >>pfl<< >>pih<< >>pih_Latn<< >>pis<< >>pis_Latn<< >>qlm<< >>rop<< >>sco<< >>sdz<< >>skw<< >>sli<< >>srm<< >>srm_Latn<< >>srn<< >>stl<< >>stq<< >>svc<< >>swg<< >>sxu<< >>tch<< >>tcs<< >>tgh<< >>tpi<< >>trf<< >>twd<< >>uln<< >>vel<< >>vic<< >>vls<< >>vmf<< >>wae<< >>wep<< >>wes<< >>wes_Latn<< >>wym<< >>ydd<< >>yec<< >>yid<< >>yih<< >>zea<< - **Original Model**: [opusTCv20210807_transformer-big_2022-08-11.zip](https://object.pouta.csc.fi/Tatoeba-MT-models/gmw-gmw/opusTCv20210807_transformer-big_2022-08-11.zip) - **Resources for more information:** - [OPUS-MT-train GitHub Repo](https://github.com/Helsinki-NLP/OPUS-MT-train) - More information about released models for this language pair: [OPUS-MT gmw-gmw README](https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/gmw-gmw/README.md) - [More information about MarianNMT models in the transformers library](https://huggingface.co/docs/transformers/model_doc/marian) - [Tatoeba Translation Challenge](https://github.com/Helsinki-NLP/Tatoeba-Challenge/ This is a multilingual translation model with multiple target languages. A sentence initial language token is required in the form of `>>id<<` (id = valid target language ID), e.g. `>>afr<<` ## Uses This model can be used for translation and text-to-text generation. ## Risks, Limitations and Biases **CONTENT WARNING: Readers should be aware that the model is trained on various public data sets that may contain content that is disturbing, offensive, and can propagate historical and current stereotypes.** Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). ## How to Get Started With the Model A short example code: ```python from transformers import MarianMTModel, MarianTokenizer src_text = [ ">>nds<< Red keinen Quatsch.", ">>eng<< Findet ihr das nicht etwas übereilt?" ] model_name = "pytorch-models/opus-mt-tc-big-gmw-gmw" tokenizer = MarianTokenizer.from_pretrained(model_name) model = MarianMTModel.from_pretrained(model_name) translated = model.generate(**tokenizer(src_text, return_tensors="pt", padding=True)) for t in translated: print( tokenizer.decode(t, skip_special_tokens=True) ) # expected output: # Kiek ok bi: Rott. # Aren't you in a hurry? ``` You can also use OPUS-MT models with the transformers pipelines, for example: ```python from transformers import pipeline pipe = pipeline("translation", model="Helsinki-NLP/opus-mt-tc-big-gmw-gmw") print(pipe(">>nds<< Red keinen Quatsch.")) # expected output: Kiek ok bi: Rott. ``` ## Training - **Data**: opusTCv20210807 ([source](https://github.com/Helsinki-NLP/Tatoeba-Challenge)) - **Pre-processing**: SentencePiece (spm32k,spm32k) - **Model Type:** transformer-big - **Original MarianNMT Model**: [opusTCv20210807_transformer-big_2022-08-11.zip](https://object.pouta.csc.fi/Tatoeba-MT-models/gmw-gmw/opusTCv20210807_transformer-big_2022-08-11.zip) - **Training Scripts**: [GitHub Repo](https://github.com/Helsinki-NLP/OPUS-MT-train) ## Evaluation * test set translations: [opusTCv20210807_transformer-big_2022-08-11.test.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/gmw-gmw/opusTCv20210807_transformer-big_2022-08-11.test.txt) * test set scores: [opusTCv20210807_transformer-big_2022-08-11.eval.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/gmw-gmw/opusTCv20210807_transformer-big_2022-08-11.eval.txt) * benchmark results: [benchmark_results.txt](benchmark_results.txt) * benchmark output: [benchmark_translations.zip](benchmark_translations.zip) | langpair | testset | chr-F | BLEU | #sent | #words | |----------|---------|-------|-------|-------|--------| | afr-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.68633 | 50.3 | 1583 | 9105 | | afr-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.70502 | 56.4 | 1374 | 9622 | | afr-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.71500 | 55.5 | 1056 | 6710 | | deu-afr | tatoeba-test-v2020-07-28-v2021-08-07 | 0.70191 | 54.2 | 1583 | 9507 | | deu-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.57304 | 34.6 | 2500 | 20797 | | deu-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.65919 | 48.4 | 17565 | 149415 | | deu-nds | tatoeba-test-v2020-07-28-v2021-08-07 | 0.48028 | 23.2 | 9999 | 76119 | | deu-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.71366 | 54.4 | 10218 | 75208 | | deu-yid | tatoeba-test-v2020-07-28-v2021-08-07 | 9.234 | 0.4 | 853 | 5353 | | eng-afr | tatoeba-test-v2020-07-28-v2021-08-07 | 0.71940 | 56.4 | 1374 | 10314 | | eng-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.62912 | 41.8 | 17565 | 151539 | | eng-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.80136 | 66.3 | 12062 | 115099 | | eng-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.70929 | 54.3 | 12696 | 91769 | | eng-yid | tatoeba-test-v2020-07-28-v2021-08-07 | 9.648 | 0.4 | 2483 | 16388 | | fry-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.40304 | 24.5 | 220 | 1573 | | fry-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.54939 | 40.5 | 260 | 1854 | | gos-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.45302 | 25.4 | 207 | 1168 | | gos-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.37587 | 23.9 | 1154 | 5634 | | gos-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.45701 | 26.1 | 1852 | 9902 | | hrx-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.51840 | 30.0 | 471 | 2805 | | hrx-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.42778 | 29.2 | 221 | 1235 | | ltz-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.37005 | 21.0 | 347 | 2208 | | ltz-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.37764 | 30.1 | 293 | 1840 | | ltz-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.32392 | 26.4 | 292 | 1685 | | multi-multi | tatoeba-test-v2020-07-28-v2021-08-07 | 0.59400 | 40.4 | 10000 | 74505 | | nds-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.63898 | 45.5 | 9999 | 74544 | | nds-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.55112 | 38.4 | 2500 | 17584 | | nds-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.66676 | 49.8 | 1657 | 11489 | | nld-afr | tatoeba-test-v2020-07-28-v2021-08-07 | 0.76610 | 62.3 | 1056 | 6823 | | nld-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.73047 | 56.7 | 10218 | 74121 | | nld-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 0.73940 | 60.2 | 12696 | 89970 | | nld-fry | tatoeba-test-v2020-07-28-v2021-08-07 | 0.47959 | 31.0 | 260 | 1857 | | nld-nds | tatoeba-test-v2020-07-28-v2021-08-07 | 0.43743 | 20.0 | 1657 | 11711 | | nld-nld | tatoeba-test-v2020-07-28-v2021-08-07 | 0.63646 | 44.9 | 1000 | 7196 | | swg-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 0.40319 | 16.3 | 1523 | 15630 | | yid-deu | tatoeba-test-v2020-07-28-v2021-08-07 | 6.304 | 0.1 | 853 | 5172 | | yid-eng | tatoeba-test-v2020-07-28-v2021-08-07 | 3.715 | 0.1 | 2483 | 15449 | | yid-yid | tatoeba-test-v2020-07-28-v2021-08-07 | 6.596 | 0.1 | 292 | 1802 | | deu-eng | newssyscomb2009 | 0.54992 | 28.2 | 502 | 11821 | | eng-deu | newssyscomb2009 | 0.53867 | 23.2 | 502 | 11271 | | deu-eng | news-test2008 | 0.54584 | 27.2 | 2051 | 49380 | | eng-deu | news-test2008 | 0.53204 | 23.7 | 2051 | 47427 | | deu-eng | newstest2009 | 0.53749 | 25.9 | 2525 | 65402 | | eng-deu | newstest2009 | 0.53283 | 22.9 | 2525 | 62816 | | deu-eng | newstest2010 | 0.58356 | 30.6 | 2489 | 61724 | | eng-deu | newstest2010 | 0.54886 | 25.8 | 2489 | 61511 | | deu-eng | newstest2011 | 0.54883 | 26.3 | 3003 | 74681 | | eng-deu | newstest2011 | 0.52712 | 23.1 | 3003 | 72981 | | deu-eng | newstest2012 | 0.56160 | 28.5 | 3003 | 72812 | | eng-deu | newstest2012 | 0.52662 | 23.3 | 3003 | 72886 | | deu-eng | newstest2013 | 0.57770 | 31.4 | 3000 | 64505 | | eng-deu | newstest2013 | 0.55774 | 27.8 | 3000 | 63737 | | deu-eng | newstest2014-deen | 0.59826 | 33.2 | 3003 | 67337 | | eng-deu | newstest2014-deen | 0.59441 | 29.6 | 3003 | 62964 | | deu-eng | newstest2015-ende | 0.59660 | 33.4 | 2169 | 46443 | | eng-deu | newstest2015-ende | 0.59889 | 32.3 | 2169 | 44260 | | deu-eng | newstest2016-ende | 0.64736 | 39.8 | 2999 | 64126 | | eng-deu | newstest2016-ende | 0.64429 | 38.3 | 2999 | 62670 | | deu-eng | newstest2017-ende | 0.60933 | 35.2 | 3004 | 64399 | | eng-deu | newstest2017-ende | 0.59258 | 30.7 | 3004 | 61291 | | deu-eng | newstest2018-ende | 0.66796 | 42.6 | 2998 | 67013 | | eng-deu | newstest2018-ende | 0.69605 | 46.5 | 2998 | 64276 | | deu-eng | newstest2019-deen | 0.63766 | 39.8 | 2000 | 39282 | | eng-deu | newstest2019-ende | 0.66880 | 43.3 | 1997 | 48969 | ## Citation Information * Publications: [OPUS-MT – Building open translation services for the World](https://aclanthology.org/2020.eamt-1.61/) and [The Tatoeba Translation Challenge – Realistic Data Sets for Low Resource and Multilingual MT](https://aclanthology.org/2020.wmt-1.139/) (Please, cite if you use this model.) ``` @inproceedings{tiedemann-thottingal-2020-opus, title = "{OPUS}-{MT} {--} Building open translation services for the World", author = {Tiedemann, J{\"o}rg and Thottingal, Santhosh}, booktitle = "Proceedings of the 22nd Annual Conference of the European Association for Machine Translation", month = nov, year = "2020", address = "Lisboa, Portugal", publisher = "European Association for Machine Translation", url = "https://aclanthology.org/2020.eamt-1.61", pages = "479--480", } @inproceedings{tiedemann-2020-tatoeba, title = "The Tatoeba Translation Challenge {--} Realistic Data Sets for Low Resource and Multilingual {MT}", author = {Tiedemann, J{\"o}rg}, booktitle = "Proceedings of the Fifth Conference on Machine Translation", month = nov, year = "2020", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2020.wmt-1.139", pages = "1174--1182", } ``` ## Acknowledgements The work is supported by the [European Language Grid](https://www.european-language-grid.eu/) as [pilot project 2866](https://live.european-language-grid.eu/catalogue/#/resource/projects/2866), by the [FoTran project](https://www.helsinki.fi/en/researchgroups/natural-language-understanding-with-cross-lingual-grounding), funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 771113), and the [MeMAD project](https://memad.eu/), funded by the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No 780069. We are also grateful for the generous computational resources and IT infrastructure provided by [CSC -- IT Center for Science](https://www.csc.fi/), Finland. ## Model conversion info * transformers version: 4.16.2 * OPUS-MT git hash: c1980b5 * port time: Sun Oct 8 14:39:59 EEST 2023 * port machine: LM0-400-22516.local