opus-mt-tc-bible-big-deu_eng_fra_por_spa-ine

Table of Contents

• Model Details
• Uses
• Risks, Limitations and Biases
• How to Get Started With the Model
• Training
• Evaluation
• Citation Information
• Acknowledgements

Model Details

Neural machine translation model for translating from unknown (deu+eng+fra+por+spa) to Indo-European languages (ine).

This model is part of the OPUS-MT project, an effort to make neural machine translation models widely available and accessible for many languages in the world. All models are originally trained using the amazing framework of Marian NMT, an efficient NMT implementation written in pure C++. The models have been converted to pyTorch using the transformers library by huggingface. Training data is taken from OPUS and training pipelines use the procedures of OPUS-MT-train. Model Description:

• Developed by: Language Technology Research Group at the University of Helsinki
• Model Type: Translation (transformer-big)
• Release: 2024-05-30
• License: Apache-2.0
• Language(s):
  • Source Language(s): deu eng fra por spa
  • Target Language(s): acf afr aln ang anp arg asm ast awa bal bar bel ben bho bis bos bpy bre bul bzj cat cbk ces chu ckb cnr cor cos crs csb cym dan deu diq div djk drt dsb dty egl ell eng enm ext fao fas fra frm fro frp frr fry fur gbm gcf gla gle glg glk glv gos got grc gsw guj hat hbs hif hin hne hns hrv hrx hsb hwc hye hyw icr isl ita jam jdt kas kea kmr kok kri ksh kur lad lah lat lav lij lim lit lld lmo lou lrc ltz mag mai mar mfe mkd mol mwl mzn nap nds nep nld nno nob non nor npi oci ofs ori orv osp oss pal pan pap pcd pcm pdc pes pfl pih pis pli pms pnt pol por prg prs pus rhg rmy roh rom ron rop rue rup rus san scn sco sdh sgs sin skr slk slv snd spa sqi srd srm srn srp stq swe swg syl szl tcs tgk tly tpi ukr urd vec vls wae wln xcl yid zea zza
  • Valid Target Language Labels: >>acf<< >>act<< >>aee<< >>aeq<< >>afr<< >>afs<< >>aig<< >>aiq<< >>aln<< >>ang<< >>ang_Latn<< >>anp<< >>anr<< >>aoa<< >>arg<< >>ask<< >>asm<< >>ast<< >>atn<< >>avd<< >>ave<< >>awa<< >>axm<< >>bah<< >>bal<< >>bal_Latn<< >>bar<< >>bdv<< >>bel<< >>ben<< >>bfb<< >>bfy<< >>bfz<< >>bgc<< >>bgd<< >>bge<< >>bgw<< >>bha<< >>bhb<< >>bhd<< >>bhe<< >>bhh<< >>bhi<< >>bho<< >>bht<< >>bhu<< >>bis<< >>bjj<< >>bjm<< >>bjs<< >>bkk<< >>bmj<< >>bns<< >>bos_Cyrl<< >>bos_Latn<< >>bpx<< >>bpy<< >>bqi<< >>bra<< >>brc<< >>bre<< >>bsg<< >>bsh<< >>btv<< >>bul<< >>bzj<< >>bzk<< >>cat<< >>cbk<< >>cbk_Latn<< >>ccd<< >>ccp<< >>cdh<< >>cdi<< >>cdj<< >>ces<< >>chu<< >>cih<< >>cim<< >>ckb<< >>cks<< >>clh<< >>cms<< >>cnr<< >>cnr_Latn<< >>cor<< >>cos<< >>cpg<< >>cri<< >>crs<< >>csb<< >>csb_Latn<< >>ctg<< >>cym<< >>czk<< >>dan<< >>dcc<< >>dcr<< >>def<< >>deh<< >>deu<< >>dhn<< >>dho<< >>diq<< >>div<< >>djk<< >>djk_Latn<< >>dlm<< >>dmk<< >>dml<< >>doi<< >>drc<< >>drt<< >>drt_Latn<< >>dry<< >>dsb<< >>dty<< >>dub<< >>duh<< >>dum<< >>dwz<< >>egl<< >>ell<< >>emx<< >>eng<< >>enm<< >>enm_Latn<< >>esh<< >>ext<< >>fab<< >>fao<< >>fas<< >>fax<< >>fay<< >>fpe<< >>fra<< >>frc<< >>frk<< >>frm<< >>frm_Latn<< >>fro<< >>fro_Latn<< >>frp<< >>frr<< >>fry<< >>fur<< >>gas<< >>gbk<< >>gbl<< >>gbm<< >>gbz<< >>gcf<< >>gcf_Latn<< >>gcl<< >>gcr<< >>gct<< >>gdx<< >>geh<< >>ggg<< >>ghc<< >>ghr<< >>gig<< >>gjk<< >>gla<< >>gle<< >>glg<< >>glh<< >>glk<< >>glv<< >>gmh<< >>gml<< >>gmy<< >>goh<< >>gos<< >>got<< >>got_Goth<< >>goz<< >>gpe<< >>gra<< >>grc<< >>grc_Grek<< >>gsw<< >>guj<< >>gul<< >>gwc<< >>gwf<< >>gwt<< >>gyn<< >>gzi<< >>hac<< >>haj<< >>hat<< >>haz<< >>hbs<< >>hbs_Cyrl<< >>hbs_Latn<< >>hca<< >>hif<< >>hif_Latn<< >>hii<< >>hin<< >>hin_Latn<< >>hit<< >>hlb<< >>hlu<< >>hne<< >>hns<< >>hrv<< >>hrx<< >>hrx_Latn<< >>hrz<< >>hsb<< >>hwc<< >>hye<< >>hyw<< >>hyw_Armn<< >>hyw_Latn<< >>icr<< >>idb<< >>imy<< >>isk<< >>isl<< >>ist<< >>ita<< >>itk<< >>jam<< >>jdg<< >>jdt<< >>jdt_Cyrl<< >>jml<< >>jnd<< >>jns<< >>jpr<< >>jut<< >>jvd<< >>kas<< >>kas_Arab<< >>kas_Deva<< >>kbu<< >>kea<< >>keq<< >>key<< >>kfm<< >>kfr<< >>kfs<< >>kft<< >>kfu<< >>kfv<< >>kfx<< >>kfy<< >>kgn<< >>khn<< >>kho<< >>khw<< >>kjo<< >>kjv<< >>kls<< >>kmr<< >>kmv<< >>kok<< >>kra<< >>kri<< >>ksh<< >>ksy<< >>ktl<< >>kur<< >>kur_Arab<< >>kur_Cyrl<< >>kur_Latn<< >>kvx<< >>kww<< >>kxp<< >>kyw<< >>lad<< >>lad_Latn<< >>lah<< >>lat<< >>lat_Latn<< >>lav<< >>lbm<< >>lhl<< >>lij<< >>lim<< >>lit<< >>lki<< >>lld<< >>lld_Latn<< >>lmn<< >>lmo<< >>lng<< >>lou<< >>lou_Latn<< >>lrc<< >>lrl<< >>lsa<< >>lss<< >>ltz<< >>luv<< >>luz<< >>mag<< >>mai<< >>mar<< >>mby<< >>mcm<< >>mfe<< >>mga<< >>mhn<< >>mjl<< >>mjz<< >>mkb<< >>mkd<< >>mke<< >>mki<< >>mnj<< >>mol<< >>mvy<< >>mwl<< >>mwr<< >>mxi<< >>mzn<< >>mzs<< >>nag<< >>nap<< >>ndf<< >>nds<< >>nep<< >>nhh<< >>nld<< >>nli<< >>nlx<< >>nno<< >>nob<< >>noe<< >>noi<< >>non<< >>nor<< >>npi<< >>nrc<< >>nrf<< >>nrn<< >>ntz<< >>nyq<< >>obt<< >>oci<< >>odk<< >>odt<< >>ofs<< >>ofs_Latn<< >>okh<< >>olt<< >>omr<< >>oor<< >>oos<< >>ori<< >>ort<< >>oru<< >>orv<< >>orv_Cyrl<< >>osc<< >>osp<< >>osp_Latn<< >>oss<< >>osx<< >>ovd<< >>owl<< >>pal<< >>pan<< >>pan_Guru<< >>pap<< >>paq<< >>pcd<< >>pcl<< >>pcm<< >>pdc<< >>pdt<< >>peo<< >>pes<< >>pey<< >>pfl<< >>pgg<< >>phd<< >>phl<< >>phv<< >>pih<< >>pih_Latn<< >>pis<< >>pli<< >>plk<< >>pln<< >>plp<< >>plq<< >>pmh<< >>pms<< >>pnt<< >>pnt_Grek<< >>pol<< >>por<< >>pov<< >>pox<< >>prc<< >>pre<< >>prg<< >>prg_Latn<< >>prn<< >>pro<< >>prs<< >>psh<< >>psi<< >>psu<< >>pus<< >>pwr<< >>raj<< >>rat<< >>rcf<< >>rdb<< >>rei<< >>rgn<< >>rhg<< >>rhg_Latn<< >>rjs<< >>rkt<< >>rmg<< >>rmi<< >>rmq<< >>rmt<< >>rmy<< >>roh<< >>rom<< >>ron<< >>rop<< >>rtw<< >>rue<< >>ruo<< >>rup<< >>ruq<< >>rus<< >>san<< >>san_Deva<< >>saz<< >>sbn<< >>scf<< >>sck<< >>scl<< >>scn<< >>sco<< >>scx<< >>sdb<< >>sdf<< >>sdg<< >>sdh<< >>sdr<< >>sdt<< >>sdz<< >>sga<< >>sgh<< >>sgl<< >>sgr<< >>sgs<< >>sgy<< >>shd<< >>shm<< >>sin<< >>siy<< >>sjp<< >>skr<< >>skw<< >>sli<< >>slk<< >>slv<< >>smm<< >>smv<< >>smy<< >>snd<< >>snd_Arab<< >>sog<< >>soi<< >>soj<< >>spa<< >>spq<< >>spx<< >>sqi<< >>sqo<< >>srd<< >>srh<< >>srm<< >>srn<< >>srp_Cyrl<< >>srx<< >>srz<< >>ssi<< >>stl<< >>stq<< >>sts<< >>svc<< >>svm<< >>svx<< >>swe<< >>swg<< >>sxl<< >>sxu<< >>syl<< >>syl_Sylo<< >>szl<< >>tch<< >>tcs<< >>tdb<< >>tgh<< >>tgk<< >>tgk_Cyrl<< >>tgk_Latn<< >>the<< >>thl<< >>thq<< >>thr<< >>tkb<< >>tks<< >>tkt<< >>tly<< >>tly_Latn<< >>tmg<< >>tnv<< >>tov<< >>tpi<< >>tra<< >>trf<< >>trm<< >>trw<< >>tsd<< >>ttt<< >>tvy<< >>twd<< >>txb<< >>txh<< >>ukr<< >>uln<< >>urd<< >>ush<< >>vaa<< >>vaf<< >>vah<< >>vas<< >>vav<< >>vec<< >>ved<< >>vel<< >>vgr<< >>vic<< >>vkp<< >>vls<< >>vmf<< >>vmh<< >>wae<< >>wbk<< >>wbl<< >>wep<< >>wes<< >>wlm<< >>wln<< >>wne<< >>wsv<< >>wtm<< >>wym<< >>xbc<< >>xbm<< >>xcb<< >>xce<< >>xcg<< >>xcl<< >>xcl_Armn<< >>xcl_Latn<< >>xco<< >>xcr<< >>xcu<< >>xdc<< >>xfa<< >>xga<< >>xgl<< >>xka<< >>xkc<< >>xkj<< >>xkp<< >>xlc<< >>xld<< >>xlg<< >>xlp<< >>xls<< >>xlu<< >>xmk<< >>xpg<< >>xpi<< >>xpr<< >>xps<< >>xsc<< >>xsd<< >>xsv<< >>xtg<< >>xto<< >>xtq<< >>xum<< >>xve<< >>xvi<< >>xvn<< >>xxx<< >>xzm<< >>yah<< >>yai<< >>ydg<< >>yec<< >>yej<< >>yid<< >>zea<< >>zum<< >>zza<<
• Original Model: opusTCv20230926max50+bt+jhubc_transformer-big_2024-05-30.zip
• Resources for more information:
  • OPUS-MT dashboard
  • OPUS-MT-train GitHub Repo
  • More information about MarianNMT models in the transformers library
  • Tatoeba Translation Challenge
  • HPLT bilingual data v1 (as part of the Tatoeba Translation Challenge dataset)
  • A massively parallel Bible corpus

This is a multilingual translation model with multiple target languages. A sentence initial language token is required in the form of >>id<< (id = valid target language ID), e.g. >>acf<<

Uses

This model can be used for translation and text-to-text generation.

Risks, Limitations and Biases

CONTENT WARNING: Readers should be aware that the model is trained on various public data sets that may contain content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).

How to Get Started With the Model

A short example code:

from transformers import MarianMTModel, MarianTokenizer

src_text = [
    ">>acf<< Replace this with text in an accepted source language.",
    ">>zza<< This is the second sentence."
]

model_name = "pytorch-models/opus-mt-tc-bible-big-deu_eng_fra_por_spa-ine"
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)
translated = model.generate(**tokenizer(src_text, return_tensors="pt", padding=True))

for t in translated:
    print( tokenizer.decode(t, skip_special_tokens=True) )

You can also use OPUS-MT models with the transformers pipelines, for example:

from transformers import pipeline
pipe = pipeline("translation", model="Helsinki-NLP/opus-mt-tc-bible-big-deu_eng_fra_por_spa-ine")
print(pipe(">>acf<< Replace this with text in an accepted source language."))

Training

• Data: opusTCv20230926max50+bt+jhubc (source)
• Pre-processing: SentencePiece (spm32k,spm32k)
• Model Type: transformer-big
• Original MarianNMT Model: opusTCv20230926max50+bt+jhubc_transformer-big_2024-05-30.zip
• Training Scripts: GitHub Repo

Evaluation

• Model scores at the OPUS-MT dashboard
• test set translations: opusTCv20230926max50+bt+jhubc_transformer-big_2024-05-29.test.txt
• test set scores: opusTCv20230926max50+bt+jhubc_transformer-big_2024-05-29.eval.txt
• benchmark results: benchmark_results.txt
• benchmark output: benchmark_translations.zip

langpair	testset	chr-F	BLEU	#sent	#words
deu-afr	tatoeba-test-v2021-08-07	0.69843	53.9	1583	9507
deu-bel	tatoeba-test-v2021-08-07	0.55105	27.8	551	3601
deu-bul	tatoeba-test-v2021-08-07	0.64736	45.3	314	2036
deu-cat	tatoeba-test-v2021-08-07	0.60706	39.7	723	5539
deu-ces	tatoeba-test-v2021-08-07	0.60622	38.3	3490	22864
deu-dan	tatoeba-test-v2021-08-07	0.71239	54.3	9998	74644
deu-deu	tatoeba-test-v2021-08-07	0.59800	36.4	2500	20806
deu-dsb	tatoeba-test-v2021-08-07	0.40343	14.6	640	3975
deu-ell	tatoeba-test-v2021-08-07	0.62906	42.3	2500	15517
deu-eng	tatoeba-test-v2021-08-07	0.63986	45.8	17565	149462
deu-fas	tatoeba-test-v2021-08-07	0.44275	18.6	3185	24941
deu-fra	tatoeba-test-v2021-08-07	0.65864	47.4	12418	102721
deu-hrv	tatoeba-test-v2021-08-07	0.64086	42.0	782	4933
deu-hsb	tatoeba-test-v2021-08-07	0.43068	19.3	666	4195
deu-isl	tatoeba-test-v2021-08-07	0.58857	37.7	969	5951
deu-ita	tatoeba-test-v2021-08-07	0.66819	44.9	10094	75504
deu-lat	tatoeba-test-v2021-08-07	0.41791	15.1	2016	10538
deu-lit	tatoeba-test-v2021-08-07	0.61083	32.6	1115	7091
deu-ltz	tatoeba-test-v2021-08-07	0.57811	37.5	347	2206
deu-nds	tatoeba-test-v2021-08-07	0.44188	18.2	9999	76137
deu-nld	tatoeba-test-v2021-08-07	0.69440	51.6	10218	75235
deu-nob	tatoeba-test-v2021-08-07	0.69018	48.9	3525	31978
deu-nor	tatoeba-test-v2021-08-07	0.68845	48.7	3651	32928
deu-pol	tatoeba-test-v2021-08-07	0.62153	38.4	5000	31726
deu-por	tatoeba-test-v2021-08-07	0.62059	40.8	10000	81482
deu-ron	tatoeba-test-v2021-08-07	0.62094	39.7	1141	7432
deu-rus	tatoeba-test-v2021-08-07	0.62731	40.3	12800	87296
deu-spa	tatoeba-test-v2021-08-07	0.66256	46.5	10521	82570
deu-swe	tatoeba-test-v2021-08-07	0.69697	54.0	3410	22701
deu-ukr	tatoeba-test-v2021-08-07	0.60825	37.3	10319	56287
eng-afr	tatoeba-test-v2021-08-07	0.70651	55.3	1374	10317
eng-bel	tatoeba-test-v2021-08-07	0.50356	23.5	2500	16237
eng-ben	tatoeba-test-v2021-08-07	0.44552	13.6	2500	11654
eng-bos_Latn	tatoeba-test-v2021-08-07	0.64765	43.2	301	1650
eng-bul	tatoeba-test-v2021-08-07	0.64094	45.0	10000	69504
eng-cat	tatoeba-test-v2021-08-07	0.65036	45.3	1631	12344
eng-ces	tatoeba-test-v2021-08-07	0.60779	39.9	13824	91332
eng-cym	tatoeba-test-v2021-08-07	0.58768	33.7	818	6020
eng-dan	tatoeba-test-v2021-08-07	0.71901	56.7	10795	79385
eng-deu	tatoeba-test-v2021-08-07	0.60900	38.8	17565	151568
eng-ell	tatoeba-test-v2021-08-07	0.68844	47.7	10899	66884
eng-eng	tatoeba-test-v2021-08-07	0.83261	69.2	12062	115106
eng-fao	tatoeba-test-v2021-08-07	0.44702	23.3	294	1933
eng-fas	tatoeba-test-v2021-08-07	0.42166	17.8	3762	31110
eng-fra	tatoeba-test-v2021-08-07	0.66478	48.0	12681	106378
eng-fry	tatoeba-test-v2021-08-07	0.44477	21.9	220	1600
eng-gla	tatoeba-test-v2021-08-07	0.45225	22.1	955	7333
eng-gle	tatoeba-test-v2021-08-07	0.58293	38.3	1913	12142
eng-glg	tatoeba-test-v2021-08-07	0.61841	41.4	1015	7881
eng-got	tatoeba-test-v2021-08-07	1.138	0.3	202	913
eng-hin	tatoeba-test-v2021-08-07	0.50692	26.6	5000	32904
eng-hrv	tatoeba-test-v2021-08-07	0.66340	44.9	1480	9396
eng-isl	tatoeba-test-v2021-08-07	0.54222	31.9	2503	19023
eng-ita	tatoeba-test-v2021-08-07	0.70559	51.1	17320	116336
eng-lad	tatoeba-test-v2021-08-07	0.40057	19.4	768	4105
eng-lad_Latn	tatoeba-test-v2021-08-07	0.46409	22.0	672	3580
eng-lat	tatoeba-test-v2021-08-07	0.42128	15.2	10298	76510
eng-lav	tatoeba-test-v2021-08-07	0.63154	39.3	1631	9932
eng-lit	tatoeba-test-v2021-08-07	0.64557	34.7	2528	14942
eng-ltz	tatoeba-test-v2021-08-07	0.58309	39.5	293	1828
eng-mar	tatoeba-test-v2021-08-07	0.49010	19.9	10396	61140
eng-mkd	tatoeba-test-v2021-08-07	0.58369	34.7	10010	62029
eng-nds	tatoeba-test-v2021-08-07	0.44492	19.7	2500	18264
eng-nld	tatoeba-test-v2021-08-07	0.69106	51.6	12696	91796
eng-nno	tatoeba-test-v2021-08-07	0.60123	38.7	460	3428
eng-nob	tatoeba-test-v2021-08-07	0.69979	53.8	4539	36119
eng-nor	tatoeba-test-v2021-08-07	0.69136	52.5	5000	39552
eng-pes	tatoeba-test-v2021-08-07	0.42287	17.9	3757	31044
eng-pol	tatoeba-test-v2021-08-07	0.63041	40.6	10099	65592
eng-por	tatoeba-test-v2021-08-07	0.70207	50.6	13222	105265
eng-ron	tatoeba-test-v2021-08-07	0.65546	44.2	5508	40367
eng-rus	tatoeba-test-v2021-08-07	0.60525	38.2	19425	134296
eng-spa	tatoeba-test-v2021-08-07	0.70563	52.4	16583	134710
eng-sqi	tatoeba-test-v2021-08-07	0.65412	41.7	1109	8281
eng-srp_Cyrl	tatoeba-test-v2021-08-07	0.59271	38.0	1580	9152
eng-swe	tatoeba-test-v2021-08-07	0.70223	55.4	10362	68067
eng-ukr	tatoeba-test-v2021-08-07	0.57326	34.0	13127	80998
eng-urd	tatoeba-test-v2021-08-07	0.43242	16.0	1663	12155
fra-bel	tatoeba-test-v2021-08-07	0.54792	30.6	283	1703
fra-bul	tatoeba-test-v2021-08-07	0.64862	44.1	446	3286
fra-cat	tatoeba-test-v2021-08-07	0.65001	42.9	700	5342
fra-ces	tatoeba-test-v2021-08-07	0.63896	41.1	438	2699
fra-dan	tatoeba-test-v2021-08-07	0.75807	62.0	1731	11312
fra-deu	tatoeba-test-v2021-08-07	0.66141	46.4	12418	100545
fra-ell	tatoeba-test-v2021-08-07	0.68793	49.0	1506	8975
fra-eng	tatoeba-test-v2021-08-07	0.70406	54.8	12681	101754
fra-fas	tatoeba-test-v2021-08-07	0.41169	16.7	376	3217
fra-fra	tatoeba-test-v2021-08-07	0.63417	44.2	1000	7757
fra-gcf	tatoeba-test-v2021-08-07	6.252	0.1	1164	6118
fra-hbs	tatoeba-test-v2021-08-07	0.55307	25.6	474	2882
fra-hrv	tatoeba-test-v2021-08-07	0.60246	34.6	258	1629
fra-ita	tatoeba-test-v2021-08-07	0.72081	52.7	10091	62060
fra-nds	tatoeba-test-v2021-08-07	0.40878	16.1	857	5760
fra-nld	tatoeba-test-v2021-08-07	0.64926	45.7	11548	82164
fra-nob	tatoeba-test-v2021-08-07	0.72336	55.8	323	2175
fra-nor	tatoeba-test-v2021-08-07	0.71320	55.0	477	3097
fra-pcd	tatoeba-test-v2021-08-07	7.211	0.1	266	1569
fra-pol	tatoeba-test-v2021-08-07	0.60675	36.6	3087	19954
fra-por	tatoeba-test-v2021-08-07	0.69533	50.1	10518	77650
fra-ron	tatoeba-test-v2021-08-07	0.64390	43.0	1925	12252
fra-rus	tatoeba-test-v2021-08-07	0.61704	39.8	11490	70123
fra-spa	tatoeba-test-v2021-08-07	0.70601	52.3	10294	78406
fra-swe	tatoeba-test-v2021-08-07	0.73948	59.1	1407	9170
fra-ukr	tatoeba-test-v2021-08-07	0.59329	36.6	10035	54372
por-cat	tatoeba-test-v2021-08-07	0.70201	50.7	747	6149
por-dan	tatoeba-test-v2021-08-07	0.78231	64.8	873	5258
por-deu	tatoeba-test-v2021-08-07	0.66060	45.8	10000	81246
por-ell	tatoeba-test-v2021-08-07	0.70353	49.6	885	4946
por-eng	tatoeba-test-v2021-08-07	0.74714	60.7	13222	105351
por-fra	tatoeba-test-v2021-08-07	0.74205	58.4	10518	80459
por-glg	tatoeba-test-v2021-08-07	0.73178	56.6	433	3016
por-ita	tatoeba-test-v2021-08-07	0.75942	57.2	3066	24897
por-nds	tatoeba-test-v2021-08-07	0.44477	19.6	207	1292
por-nld	tatoeba-test-v2021-08-07	0.68084	49.8	2500	17816
por-nor	tatoeba-test-v2021-08-07	0.72109	55.4	481	4030
por-pol	tatoeba-test-v2021-08-07	0.64745	40.5	705	4296
por-por	tatoeba-test-v2021-08-07	0.74601	55.2	2500	19220
por-ron	tatoeba-test-v2021-08-07	0.68212	46.2	681	4521
por-rus	tatoeba-test-v2021-08-07	0.63001	40.7	10000	65326
por-spa	tatoeba-test-v2021-08-07	0.78404	63.1	10947	87335
por-swe	tatoeba-test-v2021-08-07	0.73715	57.6	320	1938
por-ukr	tatoeba-test-v2021-08-07	0.62358	39.6	3372	18933
spa-afr	tatoeba-test-v2021-08-07	0.74092	59.3	448	3044
spa-bel	tatoeba-test-v2021-08-07	0.56796	28.1	205	1259
spa-bul	tatoeba-test-v2021-08-07	0.62299	43.9	286	1707
spa-cat	tatoeba-test-v2021-08-07	0.80208	63.8	1534	12343
spa-ces	tatoeba-test-v2021-08-07	0.62203	40.2	1807	11295
spa-dan	tatoeba-test-v2021-08-07	0.71414	55.1	5000	35937
spa-deu	tatoeba-test-v2021-08-07	0.65649	45.6	10521	86430
spa-ell	tatoeba-test-v2021-08-07	0.62981	41.9	1829	10464
spa-eng	tatoeba-test-v2021-08-07	0.72410	57.2	16583	138123
spa-fra	tatoeba-test-v2021-08-07	0.71593	55.0	10294	83501
spa-gla	tatoeba-test-v2021-08-07	0.46704	22.8	289	2014
spa-glg	tatoeba-test-v2021-08-07	0.73880	57.9	2121	16581
spa-hbs	tatoeba-test-v2021-08-07	0.48593	26.1	607	3513
spa-hrv	tatoeba-test-v2021-08-07	0.66819	49.1	254	1533
spa-isl	tatoeba-test-v2021-08-07	0.53084	25.4	238	1220
spa-ita	tatoeba-test-v2021-08-07	0.75756	58.1	5000	34515
spa-lad	tatoeba-test-v2021-08-07	0.41031	20.7	276	1464
spa-lad_Latn	tatoeba-test-v2021-08-07	0.48389	24.0	239	1254
spa-lat	tatoeba-test-v2021-08-07	0.41591	13.9	3129	27685
spa-lit	tatoeba-test-v2021-08-07	0.64868	39.1	454	2352
spa-mkd	tatoeba-test-v2021-08-07	0.65406	40.8	217	1073
spa-nds	tatoeba-test-v2021-08-07	0.42437	17.3	923	5941
spa-nld	tatoeba-test-v2021-08-07	0.66981	48.2	10113	79162
spa-nob	tatoeba-test-v2021-08-07	0.74315	58.0	885	6762
spa-nor	tatoeba-test-v2021-08-07	0.73699	57.1	960	7217
spa-pol	tatoeba-test-v2021-08-07	0.63650	41.1	2544	15967
spa-por	tatoeba-test-v2021-08-07	0.76008	59.2	10947	87610
spa-ron	tatoeba-test-v2021-08-07	0.67038	45.2	1959	12503
spa-rus	tatoeba-test-v2021-08-07	0.63866	42.5	10506	69242
spa-spa	tatoeba-test-v2021-08-07	0.68295	51.5	2500	21469
spa-swe	tatoeba-test-v2021-08-07	0.70919	56.2	1351	8357
spa-ukr	tatoeba-test-v2021-08-07	0.60403	37.5	10115	54544
deu-afr	flores101-devtest	0.55623	24.1	1012	25740
deu-ast	flores101-devtest	0.49967	18.1	1012	24572
deu-ces	flores101-devtest	0.51256	22.6	1012	22101
deu-dan	flores101-devtest	0.59669	31.5	1012	24638
deu-ell	flores101-devtest	0.45960	18.3	1012	26615
deu-eng	flores101-devtest	0.64491	38.0	1012	24721
deu-hin	flores101-devtest	0.45329	18.8	1012	27743
deu-isl	flores101-devtest	0.43309	14.6	1012	22834
deu-lav	flores101-devtest	0.50901	19.7	1012	22092
deu-nld	flores101-devtest	0.54019	22.0	1012	25467
deu-spa	flores101-devtest	0.50645	22.0	1012	29199
deu-swe	flores101-devtest	0.59105	30.0	1012	23121
eng-bul	flores101-devtest	0.63884	35.6	1012	24700
eng-cat	flores101-devtest	0.64377	39.0	1012	27304
eng-fas	flores101-devtest	0.48944	20.1	1012	24986
eng-gle	flores101-devtest	0.50290	21.4	1012	28707
eng-lit	flores101-devtest	0.55453	22.7	1012	20695
eng-nld	flores101-devtest	0.56191	24.6	1012	25467
eng-pol	flores101-devtest	0.48997	17.8	1012	22520
eng-por	flores101-devtest	0.68817	45.5	1012	26519
eng-slk	flores101-devtest	0.55488	26.6	1012	22543
fra-ast	flores101-devtest	0.51897	19.8	1012	24572
fra-ces	flores101-devtest	0.50228	21.3	1012	22101
fra-hrv	flores101-devtest	0.53205	22.4	1012	22423
fra-ita	flores101-devtest	0.55420	25.7	1012	27306
fra-lav	flores101-devtest	0.50495	19.3	1012	22092
fra-mkd	flores101-devtest	0.54920	23.1	1012	24314
fra-nld	flores101-devtest	0.52923	20.4	1012	25467
fra-nob	flores101-devtest	0.53696	23.0	1012	23873
fra-oci	flores101-devtest	0.56442	26.1	1012	27305
fra-por	flores101-devtest	0.60935	34.9	1012	26519
fra-ron	flores101-devtest	0.57589	30.1	1012	26799
fra-slk	flores101-devtest	0.47768	18.8	1012	22543
fra-srp_Cyrl	flores101-devtest	0.689	0.0	1012	3
por-ast	flores101-devtest	0.53848	21.5	1012	24572
por-cym	flores101-devtest	0.50778	21.8	1012	27010
por-eng	flores101-devtest	0.68628	44.1	1012	24721
por-glg	flores101-devtest	0.56990	29.2	1012	26582
por-hrv	flores101-devtest	0.53554	22.9	1012	22423
por-ltz	flores101-devtest	0.41322	11.3	1012	25087
por-nld	flores101-devtest	0.52916	20.6	1012	25467
por-nob	flores101-devtest	0.54824	24.7	1012	23873
por-oci	flores101-devtest	0.55361	24.6	1012	27305
por-ron	flores101-devtest	0.58158	31.2	1012	26799
spa-afr	flores101-devtest	0.49705	16.3	1012	25740
spa-ces	flores101-devtest	0.45466	15.5	1012	22101
spa-ell	flores101-devtest	0.44466	16.5	1012	26615
spa-hin	flores101-devtest	0.41638	14.6	1012	27743
spa-mkd	flores101-devtest	0.50767	17.4	1012	24314
spa-nld	flores101-devtest	0.49559	15.9	1012	25467
spa-npi	flores101-devtest	0.792	0.3	1012	19762
spa-swe	flores101-devtest	0.52133	20.3	1012	23121
deu-afr	flores200-devtest	0.56031	24.6	1012	25740
deu-ast	flores200-devtest	0.50634	19.2	1012	24572
deu-ben	flores200-devtest	0.40428	7.6	1012	21155
deu-bul	flores200-devtest	0.57621	28.3	1012	24700
deu-cat	flores200-devtest	0.56694	29.3	1012	27304
deu-ces	flores200-devtest	0.51425	22.8	1012	22101
deu-cym	flores200-devtest	0.50140	20.3	1012	27010
deu-dan	flores200-devtest	0.60094	32.3	1012	24638
deu-ell	flores200-devtest	0.46718	18.9	1012	26615
deu-eng	flores200-devtest	0.65234	39.1	1012	24721
deu-fra	flores200-devtest	0.60504	33.6	1012	28343
deu-fur	flores200-devtest	0.42889	14.4	1012	29171
deu-gle	flores200-devtest	0.45323	15.6	1012	28707
deu-glg	flores200-devtest	0.53882	25.8	1012	26582
deu-hat	flores200-devtest	0.46096	15.8	1012	25833
deu-hin	flores200-devtest	0.46316	19.7	1012	27743
deu-hrv	flores200-devtest	0.53819	23.0	1012	22423
deu-isl	flores200-devtest	0.44322	15.2	1012	22834
deu-ita	flores200-devtest	0.53550	23.3	1012	27306
deu-kas_Arab	flores200-devtest	3.310	0.1	1012	23514
deu-lit	flores200-devtest	0.51743	19.0	1012	20695
deu-ltz	flores200-devtest	0.50960	17.1	1012	25087
deu-mkd	flores200-devtest	0.55622	24.3	1012	24314
deu-nld	flores200-devtest	0.54434	22.7	1012	25467
deu-nno	flores200-devtest	0.51343	21.2	1012	24316
deu-nob	flores200-devtest	0.54538	23.7	1012	23873
deu-npi	flores200-devtest	0.916	0.4	1012	19762
deu-oci	flores200-devtest	0.50121	19.4	1012	27305
deu-pap	flores200-devtest	0.46836	19.1	1012	28016
deu-pol	flores200-devtest	0.47290	16.8	1012	22520
deu-por	flores200-devtest	0.58766	31.8	1012	26519
deu-prs	flores200-devtest	0.42261	15.1	1012	25885
deu-ron	flores200-devtest	0.55715	27.7	1012	26799
deu-rus	flores200-devtest	0.51013	21.9	1012	23295
deu-san	flores200-devtest	5.576	0.0	1012	18253
deu-slk	flores200-devtest	0.49473	20.5	1012	22543
deu-slv	flores200-devtest	0.53054	23.3	1012	23425
deu-spa	flores200-devtest	0.51043	22.5	1012	29199
deu-srd	flores200-devtest	0.42730	12.7	1012	28322
deu-srp_Cyrl	flores200-devtest	0.53885	24.3	1012	23456
deu-swe	flores200-devtest	0.59804	31.0	1012	23121
deu-ukr	flores200-devtest	0.50376	20.2	1012	22810
eng-afr	flores200-devtest	0.65906	37.6	1012	25740
eng-ast	flores200-devtest	0.57438	25.8	1012	24572
eng-ben	flores200-devtest	0.45263	11.1	1012	21155
eng-bul	flores200-devtest	0.64840	36.8	1012	24700
eng-cat	flores200-devtest	0.64881	39.9	1012	27304
eng-ces	flores200-devtest	0.56304	28.1	1012	22101
eng-cym	flores200-devtest	0.62000	35.5	1012	27010
eng-dan	flores200-devtest	0.68128	44.0	1012	24638
eng-deu	flores200-devtest	0.62791	34.7	1012	25094
eng-ell	flores200-devtest	0.51169	24.3	1012	26615
eng-fra	flores200-devtest	0.68379	45.2	1012	28343
eng-fur	flores200-devtest	0.44532	17.2	1012	29171
eng-gla	flores200-devtest	0.40757	11.0	1012	29377
eng-gle	flores200-devtest	0.51295	22.6	1012	28707
eng-glg	flores200-devtest	0.60361	32.9	1012	26582
eng-guj	flores200-devtest	0.44101	13.1	1012	23840
eng-hat	flores200-devtest	0.50780	21.1	1012	25833
eng-hin	flores200-devtest	0.54146	28.4	1012	27743
eng-hne	flores200-devtest	0.44142	15.7	1012	26582
eng-hrv	flores200-devtest	0.58545	28.6	1012	22423
eng-hye	flores200-devtest	0.42244	8.8	1012	22170
eng-isl	flores200-devtest	0.47667	18.9	1012	22834
eng-ita	flores200-devtest	0.57513	27.6	1012	27306
eng-kas_Arab	flores200-devtest	4.438	0.3	1012	23514
eng-lit	flores200-devtest	0.55949	23.1	1012	20695
eng-ltz	flores200-devtest	0.50438	17.0	1012	25087
eng-mag	flores200-devtest	0.46757	16.6	1012	26516
eng-mar	flores200-devtest	0.42102	9.6	1012	21810
eng-mkd	flores200-devtest	0.61359	31.3	1012	24314
eng-nld	flores200-devtest	0.56603	24.9	1012	25467
eng-nno	flores200-devtest	0.58795	29.5	1012	24316
eng-nob	flores200-devtest	0.61824	32.9	1012	23873
eng-npi	flores200-devtest	0.952	0.4	1012	19762
eng-oci	flores200-devtest	0.60388	30.7	1012	27305
eng-pap	flores200-devtest	0.53203	25.5	1012	28016
eng-pes	flores200-devtest	0.45532	17.7	1012	24986
eng-pol	flores200-devtest	0.49411	18.5	1012	22520
eng-por	flores200-devtest	0.69369	46.4	1012	26519
eng-prs	flores200-devtest	0.50119	23.1	1012	25885
eng-ron	flores200-devtest	0.62885	37.2	1012	26799
eng-rus	flores200-devtest	0.55363	26.9	1012	23295
eng-slk	flores200-devtest	0.56631	27.9	1012	22543
eng-slv	flores200-devtest	0.55956	26.6	1012	23425
eng-spa	flores200-devtest	0.54596	25.7	1012	29199
eng-srd	flores200-devtest	0.45435	16.0	1012	28322
eng-srp_Cyrl	flores200-devtest	0.60270	31.9	1012	23456
eng-swe	flores200-devtest	0.67654	42.3	1012	23121
eng-tgk	flores200-devtest	0.41975	13.0	1012	25530
eng-ukr	flores200-devtest	0.55341	25.5	1012	22810
eng-urd	flores200-devtest	0.43298	16.3	1012	28098
fra-afr	flores200-devtest	0.55327	24.5	1012	25740
fra-ast	flores200-devtest	0.52530	20.5	1012	24572
fra-ben	flores200-devtest	0.40164	7.2	1012	21155
fra-bul	flores200-devtest	0.57742	28.3	1012	24700
fra-cat	flores200-devtest	0.59840	33.6	1012	27304
fra-ces	flores200-devtest	0.50851	22.1	1012	22101
fra-cym	flores200-devtest	0.50994	21.8	1012	27010
fra-dan	flores200-devtest	0.59088	31.5	1012	24638
fra-deu	flores200-devtest	0.57176	27.5	1012	25094
fra-ell	flores200-devtest	0.47978	20.4	1012	26615
fra-eng	flores200-devtest	0.66208	41.0	1012	24721
fra-fur	flores200-devtest	0.45619	16.7	1012	29171
fra-gle	flores200-devtest	0.46469	17.0	1012	28707
fra-glg	flores200-devtest	0.56615	28.9	1012	26582
fra-hat	flores200-devtest	0.51091	20.6	1012	25833
fra-hin	flores200-devtest	0.46411	20.4	1012	27743
fra-hrv	flores200-devtest	0.53985	23.5	1012	22423
fra-isl	flores200-devtest	0.43828	14.9	1012	22834
fra-ita	flores200-devtest	0.55679	26.0	1012	27306
fra-kas_Arab	flores200-devtest	3.646	0.1	1012	23514
fra-lit	flores200-devtest	0.52001	19.4	1012	20695
fra-ltz	flores200-devtest	0.46940	14.3	1012	25087
fra-mkd	flores200-devtest	0.55627	24.2	1012	24314
fra-nld	flores200-devtest	0.53308	21.0	1012	25467
fra-nno	flores200-devtest	0.50863	20.9	1012	24316
fra-nob	flores200-devtest	0.53920	23.5	1012	23873
fra-npi	flores200-devtest	0.783	0.2	1012	19762
fra-oci	flores200-devtest	0.56888	26.8	1012	27305
fra-pap	flores200-devtest	0.47545	19.6	1012	28016
fra-pol	flores200-devtest	0.47419	16.5	1012	22520
fra-por	flores200-devtest	0.61219	35.3	1012	26519
fra-prs	flores200-devtest	0.41815	14.6	1012	25885
fra-ron	flores200-devtest	0.57969	30.7	1012	26799
fra-rus	flores200-devtest	0.51130	21.8	1012	23295
fra-san	flores200-devtest	6.562	0.1	1012	18253
fra-slk	flores200-devtest	0.48379	19.3	1012	22543
fra-slv	flores200-devtest	0.52205	22.7	1012	23425
fra-spa	flores200-devtest	0.52358	23.5	1012	29199
fra-srd	flores200-devtest	0.44853	14.2	1012	28322
fra-srp_Cyrl	flores200-devtest	0.53627	24.5	1012	23456
fra-swe	flores200-devtest	0.58741	30.0	1012	23121
fra-ukr	flores200-devtest	0.50351	19.7	1012	22810
fra-vec	flores200-devtest	0.41311	8.8	1012	26257
por-afr	flores200-devtest	0.56310	25.1	1012	25740
por-ast	flores200-devtest	0.53830	21.8	1012	24572
por-bul	flores200-devtest	0.58447	29.7	1012	24700
por-cat	flores200-devtest	0.60967	35.1	1012	27304
por-ces	flores200-devtest	0.51024	22.8	1012	22101
por-cym	flores200-devtest	0.51782	22.6	1012	27010
por-dan	flores200-devtest	0.60269	33.4	1012	24638
por-deu	flores200-devtest	0.57296	28.0	1012	25094
por-ell	flores200-devtest	0.48090	20.6	1012	26615
por-eng	flores200-devtest	0.69147	44.8	1012	24721
por-fra	flores200-devtest	0.63409	38.1	1012	28343
por-fur	flores200-devtest	0.46037	16.9	1012	29171
por-gle	flores200-devtest	0.46790	17.4	1012	28707
por-glg	flores200-devtest	0.57404	29.8	1012	26582
por-hat	flores200-devtest	0.48161	17.7	1012	25833
por-hin	flores200-devtest	0.47667	21.4	1012	27743
por-hrv	flores200-devtest	0.53963	23.5	1012	22423
por-hye	flores200-devtest	0.41362	7.5	1012	22170
por-isl	flores200-devtest	0.44102	15.8	1012	22834
por-ita	flores200-devtest	0.55156	25.2	1012	27306
por-kas_Arab	flores200-devtest	3.653	0.1	1012	23514
por-lij	flores200-devtest	0.40361	10.1	1012	28625
por-lit	flores200-devtest	0.52095	19.3	1012	20695
por-ltz	flores200-devtest	0.40955	11.4	1012	25087
por-mkd	flores200-devtest	0.56282	25.3	1012	24314
por-nld	flores200-devtest	0.53255	21.0	1012	25467
por-nno	flores200-devtest	0.51883	22.2	1012	24316
por-nob	flores200-devtest	0.55147	25.1	1012	23873
por-npi	flores200-devtest	0.808	0.3	1012	19762
por-oci	flores200-devtest	0.55910	25.3	1012	27305
por-pap	flores200-devtest	0.49408	21.4	1012	28016
por-pes	flores200-devtest	0.41574	13.4	1012	24986
por-pol	flores200-devtest	0.46551	16.1	1012	22520
por-prs	flores200-devtest	0.43325	16.1	1012	25885
por-ron	flores200-devtest	0.58583	31.8	1012	26799
por-rus	flores200-devtest	0.51109	22.4	1012	23295
por-san	flores200-devtest	8.715	0.1	1012	18253
por-slk	flores200-devtest	0.48073	19.4	1012	22543
por-slv	flores200-devtest	0.52201	22.6	1012	23425
por-spa	flores200-devtest	0.53122	24.6	1012	29199
por-srd	flores200-devtest	0.46019	15.4	1012	28322
por-srp_Cyrl	flores200-devtest	0.54401	25.6	1012	23456
por-swe	flores200-devtest	0.59310	30.7	1012	23121
por-ukr	flores200-devtest	0.51390	21.7	1012	22810
por-vec	flores200-devtest	0.41666	8.9	1012	26257
spa-afr	flores200-devtest	0.50130	16.4	1012	25740
spa-ast	flores200-devtest	0.48967	15.6	1012	24572
spa-bul	flores200-devtest	0.52657	21.6	1012	24700
spa-cat	flores200-devtest	0.53932	23.3	1012	27304
spa-ces	flores200-devtest	0.46096	16.1	1012	22101
spa-cym	flores200-devtest	0.46919	16.2	1012	27010
spa-dan	flores200-devtest	0.53445	23.2	1012	24638
spa-deu	flores200-devtest	0.51460	19.1	1012	25094
spa-ell	flores200-devtest	0.44703	16.8	1012	26615
spa-eng	flores200-devtest	0.57217	26.3	1012	24721
spa-fra	flores200-devtest	0.56332	26.0	1012	28343
spa-fur	flores200-devtest	0.42007	12.2	1012	29171
spa-gle	flores200-devtest	0.43430	13.4	1012	28707
spa-glg	flores200-devtest	0.52635	21.7	1012	26582
spa-hat	flores200-devtest	0.43928	12.9	1012	25833
spa-hin	flores200-devtest	0.42496	15.5	1012	27743
spa-hrv	flores200-devtest	0.49385	17.0	1012	22423
spa-isl	flores200-devtest	0.40834	11.8	1012	22834
spa-ita	flores200-devtest	0.52008	21.1	1012	27306
spa-kas_Arab	flores200-devtest	3.191	0.1	1012	23514
spa-lit	flores200-devtest	0.48294	14.1	1012	20695
spa-mkd	flores200-devtest	0.51381	18.6	1012	24314
spa-nld	flores200-devtest	0.50049	16.4	1012	25467
spa-nno	flores200-devtest	0.46965	16.2	1012	24316
spa-nob	flores200-devtest	0.49804	17.9	1012	23873
spa-npi	flores200-devtest	0.808	0.3	1012	19762
spa-oci	flores200-devtest	0.48255	15.6	1012	27305
spa-pap	flores200-devtest	0.43941	15.3	1012	28016
spa-pol	flores200-devtest	0.44740	13.7	1012	22520
spa-por	flores200-devtest	0.55622	25.5	1012	26519
spa-ron	flores200-devtest	0.52023	22.1	1012	26799
spa-rus	flores200-devtest	0.46740	16.6	1012	23295
spa-san	flores200-devtest	6.847	0.1	1012	18253
spa-slk	flores200-devtest	0.44274	15.0	1012	22543
spa-slv	flores200-devtest	0.48216	17.1	1012	23425
spa-srd	flores200-devtest	0.42185	11.2	1012	28322
spa-srp_Cyrl	flores200-devtest	0.48533	17.4	1012	23456
spa-swe	flores200-devtest	0.52661	20.9	1012	23121
spa-ukr	flores200-devtest	0.45966	15.0	1012	22810
deu-eng	generaltest2022	0.54326	28.7	1984	37634
deu-fra	generaltest2022	0.58009	33.7	1984	38276
eng-ces	generaltest2022	0.58390	32.4	2037	34787
eng-deu	generaltest2022	0.59050	30.9	2037	38914
eng-hrv	generaltest2022	0.51888	23.7	1671	29875
eng-rus	generaltest2022	0.53137	27.7	2037	36472
eng-ukr	generaltest2022	0.51104	22.8	2037	35418
fra-deu	generaltest2022	0.63588	39.6	2006	37696
deu-ces	multi30k_test_2016_flickr	0.49911	23.8	1000	10503
deu-eng	multi30k_test_2016_flickr	0.58821	37.5	1000	12955
deu-fra	multi30k_test_2016_flickr	0.59930	34.8	1000	13505
eng-ces	multi30k_test_2016_flickr	0.53260	26.8	1000	10503
eng-deu	multi30k_test_2016_flickr	0.62535	33.8	1000	12106
eng-fra	multi30k_test_2016_flickr	0.67525	44.9	1000	13505
fra-ces	multi30k_test_2016_flickr	0.49277	22.9	1000	10503
fra-deu	multi30k_test_2016_flickr	0.59793	30.4	1000	12106
fra-eng	multi30k_test_2016_flickr	0.64248	44.5	1000	12955
deu-eng	multi30k_test_2017_flickr	0.60958	38.5	1000	11374
deu-fra	multi30k_test_2017_flickr	0.59840	33.5	1000	12118
eng-deu	multi30k_test_2017_flickr	0.61051	32.4	1000	10755
eng-fra	multi30k_test_2017_flickr	0.68447	45.7	1000	12118
fra-deu	multi30k_test_2017_flickr	0.57580	27.4	1000	10755
fra-eng	multi30k_test_2017_flickr	0.66028	46.0	1000	11374
deu-eng	multi30k_test_2017_mscoco	0.53057	29.3	461	5231
deu-fra	multi30k_test_2017_mscoco	0.57125	31.4	461	5484
eng-deu	multi30k_test_2017_mscoco	0.56006	27.3	461	5158
eng-fra	multi30k_test_2017_mscoco	0.71206	51.5	461	5484
fra-deu	multi30k_test_2017_mscoco	0.54076	24.3	461	5158
fra-eng	multi30k_test_2017_mscoco	0.64852	45.4	461	5231
deu-ces	multi30k_test_2018_flickr	0.48008	21.6	1071	11631
deu-eng	multi30k_test_2018_flickr	0.57472	34.4	1071	14689
deu-fra	multi30k_test_2018_flickr	0.54893	27.4	1071	15867
eng-ces	multi30k_test_2018_flickr	0.54034	27.1	1071	11631
eng-deu	multi30k_test_2018_flickr	0.58338	29.0	1071	13703
eng-fra	multi30k_test_2018_flickr	0.62694	37.5	1071	15867
fra-ces	multi30k_test_2018_flickr	0.47110	19.6	1071	11631
fra-deu	multi30k_test_2018_flickr	0.53726	23.0	1071	13703
fra-eng	multi30k_test_2018_flickr	0.61916	40.2	1071	14689
eng-fra	newsdiscusstest2015	0.61964	36.0	1500	27975
fra-eng	newsdiscusstest2015	0.59754	36.1	1500	26982
deu-ces	newssyscomb2009	0.49492	21.0	502	10032
deu-eng	newssyscomb2009	0.54252	27.2	502	11818
deu-fra	newssyscomb2009	0.54517	25.3	502	12331
deu-ita	newssyscomb2009	0.54536	23.8	502	11551
deu-spa	newssyscomb2009	0.54378	26.9	502	12503
eng-ces	newssyscomb2009	0.49696	21.0	502	10032
eng-deu	newssyscomb2009	0.52782	21.8	502	11271
eng-fra	newssyscomb2009	0.57675	29.1	502	12331
eng-ita	newssyscomb2009	0.59851	29.9	502	11551
eng-spa	newssyscomb2009	0.57893	30.6	502	12503
fra-ces	newssyscomb2009	0.49843	21.0	502	10032
fra-deu	newssyscomb2009	0.52257	22.2	502	11271
fra-eng	newssyscomb2009	0.56329	29.6	502	11818
fra-ita	newssyscomb2009	0.60086	30.5	502	11551
fra-spa	newssyscomb2009	0.59982	33.7	502	12503
spa-ces	newssyscomb2009	0.50359	21.3	502	10032
spa-deu	newssyscomb2009	0.51688	20.8	502	11271
spa-eng	newssyscomb2009	0.56400	29.4	502	11818
spa-fra	newssyscomb2009	0.60759	34.0	502	12331
spa-ita	newssyscomb2009	0.60891	32.0	502	11551
deu-ces	newstest2008	0.48361	19.5	2051	42484
deu-eng	newstest2008	0.53404	25.6	2051	49380
deu-fra	newstest2008	0.53698	24.8	2051	52685
deu-spa	newstest2008	0.53079	24.8	2051	52586
eng-ces	newstest2008	0.47676	19.4	2051	42484
eng-deu	newstest2008	0.52080	22.1	2051	47447
eng-fra	newstest2008	0.54257	25.7	2051	52685
eng-spa	newstest2008	0.55703	28.8	2051	52586
fra-ces	newstest2008	0.48280	19.8	2051	42484
fra-deu	newstest2008	0.52005	21.7	2051	47447
fra-eng	newstest2008	0.53646	25.6	2051	49380
fra-spa	newstest2008	0.58093	32.6	2051	52586
spa-ces	newstest2008	0.48268	20.1	2051	42484
spa-deu	newstest2008	0.50940	20.4	2051	47447
spa-eng	newstest2008	0.54416	26.4	2051	49380
spa-fra	newstest2008	0.58071	31.6	2051	52685
deu-ces	newstest2009	0.47931	19.8	2525	55533
deu-eng	newstest2009	0.52919	24.9	2525	65399
deu-fra	newstest2009	0.52900	24.0	2525	69263
deu-ita	newstest2009	0.53248	23.2	2525	63466
deu-spa	newstest2009	0.53006	24.8	2525	68111
eng-ces	newstest2009	0.48860	20.1	2525	55533
eng-deu	newstest2009	0.52184	21.5	2525	62816
eng-fra	newstest2009	0.56544	27.9	2525	69263
eng-ita	newstest2009	0.59223	30.1	2525	63466
eng-spa	newstest2009	0.57251	29.4	2525	68111
fra-ces	newstest2009	0.48263	19.6	2525	55533
fra-deu	newstest2009	0.51645	21.5	2525	62816
fra-eng	newstest2009	0.55614	28.6	2525	65399
fra-ita	newstest2009	0.59145	30.2	2525	63466
fra-spa	newstest2009	0.58146	31.5	2525	68111
spa-ces	newstest2009	0.48619	19.9	2525	55533
spa-deu	newstest2009	0.51369	20.8	2525	62816
spa-eng	newstest2009	0.55967	28.8	2525	65399
spa-fra	newstest2009	0.58258	30.8	2525	69263
spa-ita	newstest2009	0.60212	32.2	2525	63466
deu-ces	newstest2010	0.50679	22.0	2489	52955
deu-eng	newstest2010	0.57070	29.1	2489	61711
deu-fra	newstest2010	0.56316	27.6	2489	66022
deu-spa	newstest2010	0.58086	31.1	2489	65480
eng-ces	newstest2010	0.49895	21.0	2489	52955
eng-deu	newstest2010	0.53499	23.7	2489	61503
eng-fra	newstest2010	0.58290	30.7	2489	66022
eng-spa	newstest2010	0.61355	36.1	2489	65480
fra-ces	newstest2010	0.48847	20.2	2489	52955
fra-deu	newstest2010	0.52472	22.1	2489	61503
fra-eng	newstest2010	0.57996	31.3	2489	61711
fra-spa	newstest2010	0.61617	36.8	2489	65480
spa-ces	newstest2010	0.50675	22.3	2489	52955
spa-deu	newstest2010	0.53790	24.2	2489	61503
spa-eng	newstest2010	0.60183	33.9	2489	61711
spa-fra	newstest2010	0.60950	34.4	2489	66022
deu-ces	newstest2011	0.48143	19.8	3003	65653
deu-eng	newstest2011	0.54135	25.6	3003	74681
deu-fra	newstest2011	0.54455	25.4	3003	80626
deu-spa	newstest2011	0.55696	29.2	3003	79476
eng-ces	newstest2011	0.49872	21.4	3003	65653
eng-deu	newstest2011	0.51663	21.5	3003	72981
eng-fra	newstest2011	0.59757	32.3	3003	80626
eng-spa	newstest2011	0.61259	37.2	3003	79476
fra-ces	newstest2011	0.49079	20.9	3003	65653
fra-deu	newstest2011	0.51558	21.6	3003	72981
fra-eng	newstest2011	0.58573	31.5	3003	74681
fra-spa	newstest2011	0.62198	38.7	3003	79476
spa-ces	newstest2011	0.48837	20.9	3003	65653
spa-deu	newstest2011	0.51765	22.3	3003	72981
spa-eng	newstest2011	0.58478	32.0	3003	74681
spa-fra	newstest2011	0.60332	33.6	3003	80626
deu-ces	newstest2012	0.47349	19.3	3003	65456
deu-eng	newstest2012	0.54991	27.0	3003	72812
deu-fra	newstest2012	0.54838	26.2	3003	78011
deu-rus	newstest2012	0.47331	17.8	3003	64790
deu-spa	newstest2012	0.56115	29.7	3003	79006
eng-ces	newstest2012	0.47312	19.2	3003	65456
eng-deu	newstest2012	0.51588	22.0	3003	72886
eng-fra	newstest2012	0.58096	30.1	3003	78011
eng-rus	newstest2012	0.58916	30.8	3003	64790
eng-spa	newstest2012	0.61457	37.2	3003	79006
fra-ces	newstest2012	0.47412	19.8	3003	65456
fra-deu	newstest2012	0.51511	22.3	3003	72886
fra-eng	newstest2012	0.58053	31.4	3003	72812
fra-rus	newstest2012	0.49149	20.1	3003	64790
fra-spa	newstest2012	0.61788	37.9	3003	79006
spa-ces	newstest2012	0.47397	19.8	3003	65456
spa-deu	newstest2012	0.52060	22.7	3003	72886
spa-eng	newstest2012	0.60936	35.9	3003	72812
spa-fra	newstest2012	0.60277	33.5	3003	78011
spa-rus	newstest2012	0.50113	21.2	3003	64790
deu-ces	newstest2013	0.49584	22.1	3000	57250
deu-eng	newstest2013	0.56741	30.2	3000	64505
deu-fra	newstest2013	0.54857	27.6	3000	70037
deu-rus	newstest2013	0.49961	21.3	3000	58560
deu-spa	newstest2013	0.55886	30.1	3000	70528
eng-ces	newstest2013	0.50582	23.3	3000	57250
eng-deu	newstest2013	0.54401	25.8	3000	63737
eng-fra	newstest2013	0.57648	31.4	3000	70037
eng-rus	newstest2013	0.51291	22.7	3000	58560
eng-spa	newstest2013	0.58821	33.7	3000	70528
fra-ces	newstest2013	0.48374	21.2	3000	57250
fra-deu	newstest2013	0.52935	24.2	3000	63737
fra-eng	newstest2013	0.58033	32.7	3000	64505
fra-rus	newstest2013	0.50016	21.5	3000	58560
fra-spa	newstest2013	0.58532	33.6	3000	70528
spa-ces	newstest2013	0.49954	22.8	3000	57250
spa-deu	newstest2013	0.53643	24.7	3000	63737
spa-eng	newstest2013	0.59270	33.2	3000	64505
spa-fra	newstest2013	0.58979	33.0	3000	70037
spa-rus	newstest2013	0.51659	23.3	3000	58560
deu-eng	newstest2014	0.57690	30.3	3003	67337
eng-ces	newstest2014	0.54124	25.2	3003	59902
eng-deu	newstest2014	0.56967	25.8	3003	62688
eng-fra	newstest2014	0.63245	37.3	3003	77306
eng-hin	newstest2014	0.48263	20.4	2507	60872
eng-rus	newstest2014	0.61540	33.8	3003	61603
fra-eng	newstest2014	0.61729	35.4	3003	70708
deu-eng	newstest2015	0.58051	31.5	2169	46443
eng-ces	newstest2015	0.50901	22.1	2656	45858
eng-deu	newstest2015	0.58038	29.6	2169	44260
eng-rus	newstest2015	0.56909	28.0	2818	55920
deu-eng	newstest2016	0.62651	37.1	2999	64119
eng-ces	newstest2016	0.53138	25.4	2999	56998
eng-deu	newstest2016	0.62052	35.2	2999	62669
eng-ron	newstest2016	0.57801	30.1	1999	48945
eng-rus	newstest2016	0.55239	26.3	2998	62014
deu-eng	newstest2017	0.59034	32.9	3004	64399
eng-ces	newstest2017	0.49009	20.9	3005	54361
eng-deu	newstest2017	0.57301	28.3	3004	61287
eng-lav	newstest2017	0.49647	17.3	2001	39392
eng-rus	newstest2017	0.57889	29.5	3001	60253
deu-eng	newstest2018	0.64488	39.9	2998	67012
eng-ces	newstest2018	0.49105	20.2	2983	54652
eng-deu	newstest2018	0.66353	41.7	2998	64276
eng-rus	newstest2018	0.55568	25.6	3000	61907
deu-ces	newstest2019	0.48893	20.5	1997	43113
deu-eng	newstest2019	0.61048	35.6	2000	39227
deu-fra	newstest2019	0.61083	33.4	1701	42509
eng-ces	newstest2019	0.50528	22.0	1997	43113
eng-deu	newstest2019	0.63438	37.7	1997	48746
eng-guj	newstest2019	0.46302	15.3	998	21924
eng-lit	newstest2019	0.48848	16.0	998	19711
eng-rus	newstest2019	0.53243	25.9	1997	48147
fra-deu	newstest2019	0.58722	26.9	1701	36446
deu-eng	newstest2020	0.57511	30.8	785	38220
deu-fra	newstest2020	0.58907	30.9	1619	36890
eng-deu	newstest2020	0.56905	28.2	1418	52383
eng-pol	newstest2020	0.51806	21.6	1000	23855
eng-rus	newstest2020	0.49190	20.9	2002	47083
fra-deu	newstest2020	0.58687	25.9	1619	30265
deu-eng	newstest2021	0.58074	29.6	1000	20180
deu-fra	newstest2021	0.57642	29.9	1000	23757
eng-ces	newstest2021	0.45921	15.9	1002	25589
eng-deu	newstest2021	0.54593	23.8	1002	27970
eng-isl	newstest2021	0.46926	18.6	1000	25233
eng-rus	newstest2021	0.50587	22.1	1002	26095
fra-deu	newstest2021	0.62883	36.0	1026	26077
deu-eng	newstestALL2020	0.57511	30.8	785	38220
eng-deu	newstestALL2020	0.56905	28.2	1418	52383
deu-eng	newstestB2020	0.57270	30.4	785	37696
eng-deu	newstestB2020	0.55895	27.8	1418	53092
deu-afr	ntrex128	0.55112	25.2	1997	50050
deu-bel	ntrex128	0.43275	14.7	1997	43951
deu-ben	ntrex128	0.40104	6.3	1997	40095
deu-bul	ntrex128	0.50482	22.6	1997	48832
deu-cat	ntrex128	0.53145	25.8	1997	53438
deu-ces	ntrex128	0.49340	20.9	1997	43356
deu-cym	ntrex128	0.46757	16.9	1997	51768
deu-dan	ntrex128	0.54629	27.2	1997	47643
deu-div	ntrex128	9.726	0.0	1997	37802
deu-ell	ntrex128	0.47825	21.0	1997	52028
deu-eng	ntrex128	0.60047	31.8	1997	47673
deu-fas	ntrex128	0.40193	12.7	1997	50525
deu-fra	ntrex128	0.54175	26.6	1997	53481
deu-gle	ntrex128	0.42902	13.7	1997	55846
deu-glg	ntrex128	0.53430	25.5	1997	50432
deu-hin	ntrex128	0.41309	15.0	1997	55219
deu-hrv	ntrex128	0.51998	22.0	1997	44449
deu-isl	ntrex128	0.44649	16.0	1997	46643
deu-ita	ntrex128	0.54145	24.8	1997	50759
deu-lav	ntrex128	0.45243	14.0	1997	44709
deu-lit	ntrex128	0.47694	15.1	1997	41189
deu-ltz	ntrex128	0.45326	13.1	1997	49763
deu-mkd	ntrex128	0.52189	21.8	1997	47552
deu-nld	ntrex128	0.55518	26.0	1997	51884
deu-nno	ntrex128	0.49799	20.2	1997	46512
deu-nob	ntrex128	0.54700	25.7	1997	45501
deu-pol	ntrex128	0.47430	18.1	1997	44700
deu-por	ntrex128	0.53262	24.6	1997	51631
deu-ron	ntrex128	0.51091	24.2	1997	53498
deu-rus	ntrex128	0.47374	20.0	1997	45714
deu-slk	ntrex128	0.47850	19.9	1997	44242
deu-slv	ntrex128	0.50797	21.7	1997	46072
deu-snd_Arab	ntrex128	0.646	0.1	1997	49866
deu-spa	ntrex128	0.55580	28.9	1997	54107
deu-sqi	ntrex128	0.50383	21.4	1997	54563
deu-swe	ntrex128	0.56144	26.9	1997	44889
deu-ukr	ntrex128	0.45481	16.5	1997	45346
eng-afr	ntrex128	0.66106	40.7	1997	50050
eng-bel	ntrex128	0.47642	19.3	1997	43951
eng-ben	ntrex128	0.44938	10.2	1997	40095
eng-bul	ntrex128	0.55238	28.3	1997	48832
eng-cat	ntrex128	0.59563	33.8	1997	53438
eng-ces	ntrex128	0.54586	26.4	1997	43356
eng-cym	ntrex128	0.54759	26.0	1997	51768
eng-dan	ntrex128	0.60350	35.6	1997	47643
eng-deu	ntrex128	0.57027	27.9	1997	48761
eng-div	ntrex128	9.103	0.0	1997	37802
eng-ell	ntrex128	0.53745	28.5	1997	52028
eng-fas	ntrex128	0.44487	17.0	1997	50525
eng-fra	ntrex128	0.59630	32.9	1997	53481
eng-gle	ntrex128	0.48192	19.0	1997	55846
eng-glg	ntrex128	0.60389	34.3	1997	50432
eng-guj	ntrex128	0.40892	10.6	1997	45335
eng-hin	ntrex128	0.48575	22.9	1997	55219
eng-hrv	ntrex128	0.58236	30.0	1997	44449
eng-isl	ntrex128	0.48066	20.3	1997	46643
eng-ita	ntrex128	0.60485	33.7	1997	50759
eng-lav	ntrex128	0.49041	17.5	1997	44709
eng-lit	ntrex128	0.51230	18.3	1997	41189
eng-ltz	ntrex128	0.46754	15.0	1997	49763
eng-mkd	ntrex128	0.59421	30.5	1997	47552
eng-nld	ntrex128	0.59539	31.7	1997	51884
eng-nno	ntrex128	0.58757	30.9	1997	46512
eng-nob	ntrex128	0.61045	35.2	1997	45501
eng-pol	ntrex128	0.50306	21.5	1997	44700
eng-por	ntrex128	0.59064	32.3	1997	51631
eng-prs	ntrex128	0.40486	13.7	1997	52316
eng-ron	ntrex128	0.56071	30.3	1997	53498
eng-rus	ntrex128	0.52359	26.5	1997	45714
eng-slk	ntrex128	0.54391	26.5	1997	44242
eng-slv	ntrex128	0.54372	25.9	1997	46072
eng-snd_Arab	ntrex128	0.659	0.1	1997	49866
eng-spa	ntrex128	0.62654	38.1	1997	54107
eng-sqi	ntrex128	0.56845	30.7	1997	54563
eng-srp_Cyrl	ntrex128	0.43244	18.5	1997	45487
eng-swe	ntrex128	0.64175	38.8	1997	44889
eng-ukr	ntrex128	0.49480	21.0	1997	45346
eng-urd	ntrex128	0.44382	17.8	1997	54259
fra-afr	ntrex128	0.53285	23.8	1997	50050
fra-bel	ntrex128	0.42933	14.5	1997	43951
fra-bul	ntrex128	0.50188	22.6	1997	48832
fra-cat	ntrex128	0.54019	26.5	1997	53438
fra-ces	ntrex128	0.47876	19.4	1997	43356
fra-cym	ntrex128	0.47101	18.2	1997	51768
fra-dan	ntrex128	0.52518	25.5	1997	47643
fra-deu	ntrex128	0.51567	21.8	1997	48761
fra-div	ntrex128	9.433	0.0	1997	37802
fra-ell	ntrex128	0.47986	21.3	1997	52028
fra-eng	ntrex128	0.59200	31.6	1997	47673
fra-gle	ntrex128	0.42837	13.5	1997	55846
fra-glg	ntrex128	0.54310	26.4	1997	50432
fra-hin	ntrex128	0.41234	15.4	1997	55219
fra-hrv	ntrex128	0.51276	21.4	1997	44449
fra-isl	ntrex128	0.44117	16.4	1997	46643
fra-ita	ntrex128	0.55460	26.5	1997	50759
fra-lav	ntrex128	0.44057	13.3	1997	44709
fra-lit	ntrex128	0.46745	13.9	1997	41189
fra-ltz	ntrex128	0.42060	11.2	1997	49763
fra-mkd	ntrex128	0.51077	20.7	1997	47552
fra-nld	ntrex128	0.53226	23.6	1997	51884
fra-nno	ntrex128	0.48806	19.5	1997	46512
fra-nob	ntrex128	0.52610	24.2	1997	45501
fra-pol	ntrex128	0.46942	17.9	1997	44700
fra-por	ntrex128	0.53724	25.2	1997	51631
fra-ron	ntrex128	0.51064	23.7	1997	53498
fra-rus	ntrex128	0.47588	20.7	1997	45714
fra-slk	ntrex128	0.46230	18.1	1997	44242
fra-slv	ntrex128	0.49389	20.4	1997	46072
fra-snd_Arab	ntrex128	0.494	0.1	1997	49866
fra-spa	ntrex128	0.56432	30.0	1997	54107
fra-sqi	ntrex128	0.50181	21.7	1997	54563
fra-swe	ntrex128	0.54462	25.8	1997	44889
fra-ukr	ntrex128	0.44589	15.8	1997	45346
por-afr	ntrex128	0.56124	26.6	1997	50050
por-bel	ntrex128	0.44346	15.4	1997	43951
por-bul	ntrex128	0.51578	24.1	1997	48832
por-cat	ntrex128	0.57052	30.4	1997	53438
por-ces	ntrex128	0.49385	21.0	1997	43356
por-cym	ntrex128	0.47606	18.0	1997	51768
por-dan	ntrex128	0.54564	27.9	1997	47643
por-deu	ntrex128	0.53023	22.6	1997	48761
por-div	ntrex128	8.442	0.0	1997	37802
por-ell	ntrex128	0.50229	24.0	1997	52028
por-eng	ntrex128	0.62761	36.2	1997	47673
por-fas	ntrex128	0.40704	13.2	1997	50525
por-fra	ntrex128	0.55864	27.9	1997	53481
por-gle	ntrex128	0.43855	14.4	1997	55846
por-glg	ntrex128	0.56576	28.9	1997	50432
por-hin	ntrex128	0.41906	16.1	1997	55219
por-hrv	ntrex128	0.52756	22.9	1997	44449
por-isl	ntrex128	0.44649	16.4	1997	46643
por-ita	ntrex128	0.57836	29.5	1997	50759
por-lav	ntrex128	0.45519	14.3	1997	44709
por-lit	ntrex128	0.48342	15.2	1997	41189
por-mkd	ntrex128	0.53728	23.3	1997	47552
por-nld	ntrex128	0.54708	25.9	1997	51884
por-nno	ntrex128	0.50133	20.9	1997	46512
por-nob	ntrex128	0.54357	25.6	1997	45501
por-pol	ntrex128	0.47970	18.5	1997	44700
por-ron	ntrex128	0.53061	26.3	1997	53498
por-rus	ntrex128	0.48075	21.3	1997	45714
por-slk	ntrex128	0.47540	19.2	1997	44242
por-slv	ntrex128	0.50780	21.9	1997	46072
por-snd_Arab	ntrex128	0.565	0.1	1997	49866
por-spa	ntrex128	0.58969	33.3	1997	54107
por-sqi	ntrex128	0.52446	25.0	1997	54563
por-srp_Cyrl	ntrex128	0.40145	14.6	1997	45487
por-swe	ntrex128	0.55721	26.9	1997	44889
por-ukr	ntrex128	0.46398	17.9	1997	45346
spa-afr	ntrex128	0.55892	25.5	1997	50050
spa-bel	ntrex128	0.44316	15.1	1997	43951
spa-ben	ntrex128	0.40604	6.6	1997	40095
spa-bul	ntrex128	0.51341	23.6	1997	48832
spa-cat	ntrex128	0.60359	33.7	1997	53438
spa-ces	ntrex128	0.49078	20.2	1997	43356
spa-cym	ntrex128	0.48640	19.0	1997	51768
spa-dan	ntrex128	0.54368	26.6	1997	47643
spa-deu	ntrex128	0.52793	22.0	1997	48761
spa-div	ntrex128	8.405	0.0	1997	37802
spa-ell	ntrex128	0.49688	23.5	1997	52028
spa-eng	ntrex128	0.62576	34.6	1997	47673
spa-fas	ntrex128	0.41004	13.0	1997	50525
spa-fra	ntrex128	0.56440	28.5	1997	53481
spa-gle	ntrex128	0.44229	14.7	1997	55846
spa-glg	ntrex128	0.60955	33.9	1997	50432
spa-hin	ntrex128	0.42185	16.1	1997	55219
spa-hrv	ntrex128	0.53061	22.8	1997	44449
spa-isl	ntrex128	0.45185	17.1	1997	46643
spa-ita	ntrex128	0.56945	27.9	1997	50759
spa-lav	ntrex128	0.45424	14.2	1997	44709
spa-lit	ntrex128	0.48405	15.5	1997	41189
spa-mkd	ntrex128	0.53653	22.9	1997	47552
spa-nld	ntrex128	0.54795	25.6	1997	51884
spa-nno	ntrex128	0.50373	20.9	1997	46512
spa-nob	ntrex128	0.54331	25.4	1997	45501
spa-pol	ntrex128	0.47592	18.3	1997	44700
spa-por	ntrex128	0.56627	28.8	1997	51631
spa-ron	ntrex128	0.53162	26.5	1997	53498
spa-rus	ntrex128	0.48366	21.3	1997	45714
spa-slk	ntrex128	0.47742	19.5	1997	44242
spa-slv	ntrex128	0.50955	21.5	1997	46072
spa-snd_Arab	ntrex128	0.611	0.1	1997	49866
spa-sqi	ntrex128	0.52450	24.1	1997	54563
spa-srp_Cyrl	ntrex128	0.40529	14.8	1997	45487
spa-swe	ntrex128	0.55918	26.5	1997	44889
spa-ukr	ntrex128	0.46554	18.0	1997	45346
eng-ben	tico19-test	0.43061	12.1	2100	51695
eng-fas	tico19-test	0.49470	21.5	2100	59779
eng-fra	tico19-test	0.60842	37.1	2100	64661
eng-hin	tico19-test	0.56535	33.7	2100	62680
eng-nep	tico19-test	0.43865	10.6	2100	48363
eng-por	tico19-test	0.70681	46.5	2100	62729
eng-prs	tico19-test	0.42390	16.0	2100	62972
eng-rus	tico19-test	0.54724	27.3	2100	55843
eng-spa	tico19-test	0.70628	47.9	2100	66563
eng-urd	tico19-test	0.44071	16.9	2100	65312
fra-eng	tico19-test	0.60880	37.3	2100	56323
fra-fas	tico19-test	0.41606	15.7	2100	59779
fra-hin	tico19-test	0.45682	22.6	2100	62680
fra-por	tico19-test	0.57865	33.3	2100	62729
fra-rus	tico19-test	0.47330	21.3	2100	55843
fra-spa	tico19-test	0.58605	35.3	2100	66563
por-eng	tico19-test	0.72407	49.2	2100	56315
por-fas	tico19-test	0.46862	18.3	2100	59779
por-fra	tico19-test	0.58796	33.5	2100	64661
por-hin	tico19-test	0.52110	27.5	2100	62680
por-nep	tico19-test	0.41083	7.9	2100	48363
por-rus	tico19-test	0.52261	24.0	2100	55843
por-spa	tico19-test	0.67491	43.7	2100	66563
por-urd	tico19-test	0.41511	14.4	2100	65312
spa-ben	tico19-test	0.40476	9.3	2100	51695
spa-eng	tico19-test	0.70962	47.1	2100	56315
spa-fas	tico19-test	0.46948	18.4	2100	59779
spa-fra	tico19-test	0.59250	34.3	2100	64661
spa-hin	tico19-test	0.51847	27.4	2100	62680
spa-nep	tico19-test	0.40390	7.3	2100	48363
spa-por	tico19-test	0.66969	41.5	2100	62729
spa-prs	tico19-test	0.40661	13.3	2100	62972
spa-rus	tico19-test	0.53338	25.0	2100	55843
spa-urd	tico19-test	0.41128	13.9	2100	65312

Citation Information

• Publications: Democratizing neural machine translation with OPUS-MT and OPUS-MT – Building open translation services for the World and The Tatoeba Translation Challenge – Realistic Data Sets for Low Resource and Multilingual MT (Please, cite if you use this model.)

@article{tiedemann2023democratizing,
  title={Democratizing neural machine translation with {OPUS-MT}},
  author={Tiedemann, J{\"o}rg and Aulamo, Mikko and Bakshandaeva, Daria and Boggia, Michele and Gr{\"o}nroos, Stig-Arne and Nieminen, Tommi and Raganato, Alessandro and Scherrer, Yves and Vazquez, Raul and Virpioja, Sami},
  journal={Language Resources and Evaluation},
  number={58},
  pages={713--755},
  year={2023},
  publisher={Springer Nature},
  issn={1574-0218},
  doi={10.1007/s10579-023-09704-w}
}

@inproceedings{tiedemann-thottingal-2020-opus,
    title = "{OPUS}-{MT} {--} Building open translation services for the World",
    author = {Tiedemann, J{\"o}rg  and Thottingal, Santhosh},
    booktitle = "Proceedings of the 22nd Annual Conference of the European Association for Machine Translation",
    month = nov,
    year = "2020",
    address = "Lisboa, Portugal",
    publisher = "European Association for Machine Translation",
    url = "https://aclanthology.org/2020.eamt-1.61",
    pages = "479--480",
}

@inproceedings{tiedemann-2020-tatoeba,
    title = "The Tatoeba Translation Challenge {--} Realistic Data Sets for Low Resource and Multilingual {MT}",
    author = {Tiedemann, J{\"o}rg},
    booktitle = "Proceedings of the Fifth Conference on Machine Translation",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.wmt-1.139",
    pages = "1174--1182",
}

Acknowledgements

The work is supported by the HPLT project, funded by the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070350. We are also grateful for the generous computational resources and IT infrastructure provided by CSC -- IT Center for Science, Finland, and the EuroHPC supercomputer LUMI.

Model conversion info

• transformers version: 4.45.1
• OPUS-MT git hash: 0882077
• port time: Tue Oct 8 10:12:36 EEST 2024
• port machine: LM0-400-22516.local