utrobinmv commited on
Commit
189b09b
1 Parent(s): c4bf1ac

feat add app

Browse files
Files changed (2) hide show
  1. app.py +20 -0
  2. table.html +856 -0
app.py ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import gradio as gr
3
+
4
+ def load_csv():
5
+ filename = 'table.html'
6
+ with open(filename, 'r', encoding="utf-8-sig") as f:
7
+ return f.read()
8
+ #return df.to_html(render_links=True)
9
+ return ''
10
+
11
+ iface = gr.Interface(
12
+ fn=load_csv,
13
+ inputs=None,
14
+ outputs="html",
15
+ title="NTREX dataset Benchmark ",
16
+ description="Benchmark translation to TREX dataset for ru, en, zh languages. Dataset: https://github.com/MicrosoftTranslator/NTREX",
17
+ live=True
18
+ )
19
+
20
+ iface.launch(share=False)
table.html ADDED
@@ -0,0 +1,856 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <table border="1" class="dataframe">
2
+ <thead>
3
+ <tr style="text-align: right;">
4
+ <th>name</th>
5
+ <th>pair</th>
6
+ <th>URL</th>
7
+ <th>params</th>
8
+ <th>bleu</th>
9
+ <th>chrf</th>
10
+ <th>bertscore_f1</th>
11
+ <th>ter</th>
12
+ <th>meteor</th>
13
+ <th>rouge1</th>
14
+ <th>rouge2</th>
15
+ <th>rougeL</th>
16
+ <th>rougeLsum</th>
17
+ <th>CPU</th>
18
+ <th>GPU</th>
19
+ </tr>
20
+ </thead>
21
+ <tbody>
22
+ <tr>
23
+ <td>Helsinki-NLP/opus-mt-en-ru</td>
24
+ <td>en-ru</td>
25
+ <td><a href="https://huggingface.co/Helsinki-NLP/opus-mt-en-ru" target="_blank">open</a></td>
26
+ <td>73M</td>
27
+ <td>25,0095</td>
28
+ <td>51,4155</td>
29
+ <td>0,8476</td>
30
+ <td>66,4672</td>
31
+ <td>0,4722</td>
32
+ <td>0,2224</td>
33
+ <td>0,0879</td>
34
+ <td>0,2197</td>
35
+ <td>0,2198</td>
36
+ <td>429,3787</td>
37
+ <td>178,6283</td>
38
+ </tr>
39
+ <tr>
40
+ <td>facebook/m2m100_418M</td>
41
+ <td>en-ru</td>
42
+ <td><a href="https://huggingface.co/facebook/m2m100_418M" target="_blank">open</a></td>
43
+ <td>461M</td>
44
+ <td>22,0757</td>
45
+ <td>48,4266</td>
46
+ <td>0,8408</td>
47
+ <td>68,4320</td>
48
+ <td>0,4529</td>
49
+ <td>0,2489</td>
50
+ <td>0,1016</td>
51
+ <td>0,2456</td>
52
+ <td>0,2456</td>
53
+ <td>2601,0673</td>
54
+ <td>461,8845</td>
55
+ </tr>
56
+ <tr>
57
+ <td>facebook/mbart-large-50-many-to-many-mmt</td>
58
+ <td>en-ru</td>
59
+ <td><a href="https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt" target="_blank">open</a></td>
60
+ <td>582M</td>
61
+ <td>22,8219</td>
62
+ <td>49,3038</td>
63
+ <td>0,8423</td>
64
+ <td>68,4744</td>
65
+ <td>0,4624</td>
66
+ <td>0,2428</td>
67
+ <td>0,1049</td>
68
+ <td>0,2390</td>
69
+ <td>0,2392</td>
70
+ <td>2686,7609</td>
71
+ <td>505,1522</td>
72
+ </tr>
73
+ <tr>
74
+ <td>facebook/nllb-200-distilled-1.3B</td>
75
+ <td>en-ru</td>
76
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-1.3B" target="_blank">open</a></td>
77
+ <td>1307M</td>
78
+ <td>26,3921</td>
79
+ <td>53,4754</td>
80
+ <td><span style="font-weight: bold;background-color: Lime;">0,8590</td>
81
+ <td>62,6019</td>
82
+ <td>0,4957</td>
83
+ <td>0,2563</td>
84
+ <td><span style="font-weight: bold;background-color: Lime;">0,1118</td>
85
+ <td>0,2537</td>
86
+ <td>0,2537</td>
87
+ <td>29748,2773</td>
88
+ <td>742,6220</td>
89
+ </tr>
90
+ <tr>
91
+ <td>facebook/nllb-200-distilled-600M</td>
92
+ <td>en-ru</td>
93
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-600M" target="_blank">open</a></td>
94
+ <td>586M</td>
95
+ <td>25,2193</td>
96
+ <td>52,0565</td>
97
+ <td>0,8541</td>
98
+ <td>64,9225</td>
99
+ <td>0,4699</td>
100
+ <td><span style="font-weight: bold;background-color: Lime;">0,2612</td>
101
+ <td>0,1113</td>
102
+ <td><span style="font-weight: bold;background-color: Lime;">0,2585</td>
103
+ <td><span style="font-weight: bold;background-color: Lime;">0,2586</td>
104
+ <td>2138,9623</td>
105
+ <td>523,5301</td>
106
+ </tr>
107
+ <tr>
108
+ <td>facebook/wmt19-en-ru</td>
109
+ <td>en-ru</td>
110
+ <td><a href="https://huggingface.co/facebook/wmt19-en-ru" target="_blank">open</a></td>
111
+ <td>279M</td>
112
+ <td>27,4767</td>
113
+ <td>53,6640</td>
114
+ <td>0,8569</td>
115
+ <td>63,9424</td>
116
+ <td>0,4961</td>
117
+ <td>0,2514</td>
118
+ <td>0,1076</td>
119
+ <td>0,2489</td>
120
+ <td>0,2488</td>
121
+ <td>924,8711</td>
122
+ <td>206,3258</td>
123
+ </tr>
124
+ <tr>
125
+ <td>utrobinmv/t5_translate_en_ru_zh_base_200</td>
126
+ <td>en-ru</td>
127
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_base_200" target="_blank">open</a></span></td>
128
+ <td>284M</td>
129
+ <td><span style="font-weight: bold;background-color: Lime;">28,5759</td>
130
+ <td>54,2799</td>
131
+ <td>0,8554</td>
132
+ <td>62,4948</td>
133
+ <td>0,5174</td>
134
+ <td>0,1908</td>
135
+ <td>0,0655</td>
136
+ <td>0,1895</td>
137
+ <td>0,1893</td>
138
+ <td>1171,5537</td>
139
+ <td>502,5175</td>
140
+ </tr>
141
+ <tr>
142
+ <td>utrobinmv/t5_translate_en_ru_zh_large_1024</td>
143
+ <td>en-ru</td>
144
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024" target="_blank">open</a></span></td>
145
+ <td>811M</td>
146
+ <td>28,0225</td>
147
+ <td><span style="font-weight: bold;background-color: Lime;">54,5335</td>
148
+ <td>0,8586</td>
149
+ <td>62,6889</td>
150
+ <td><span style="font-weight: bold;background-color: Lime;">0,5272</td>
151
+ <td>0,1932</td>
152
+ <td>0,0676</td>
153
+ <td>0,1913</td>
154
+ <td>0,1913</td>
155
+ <td>56570,4151</td>
156
+ <td>988,1287</td>
157
+ </tr>
158
+ <tr>
159
+ <td>utrobinmv/t5_translate_en_ru_zh_small_1024</td>
160
+ <td>en-ru</td>
161
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_small_1024" target="_blank">open</a></span></td>
162
+ <td>105M</td>
163
+ <td>19,2178</td>
164
+ <td>46,3500</td>
165
+ <td>0,8257</td>
166
+ <td><span style="font-weight: bold;background-color: Lime;">71,6757</td>
167
+ <td>0,4241</td>
168
+ <td>0,1970</td>
169
+ <td>0,0678</td>
170
+ <td>0,1950</td>
171
+ <td>0,1950</td>
172
+ <td>736,5292</td>
173
+ <td>319,2867</td>
174
+ </tr>
175
+ <tr>
176
+ <td>Helsinki-NLP/opus-mt-en-zh</td>
177
+ <td>en-zh</td>
178
+ <td><a href="https://huggingface.co/Helsinki-NLP/opus-mt-en-zh" target="_blank">open</a></td>
179
+ <td>74M</td>
180
+ <td>1,4242</td>
181
+ <td>23,8306</td>
182
+ <td>0,7899</td>
183
+ <td>159,1038</td>
184
+ <td>0,0478</td>
185
+ <td>0,2227</td>
186
+ <td>0,0962</td>
187
+ <td>0,2198</td>
188
+ <td>0,2200</td>
189
+ <td>338,5790</td>
190
+ <td>134,3619</td>
191
+ </tr>
192
+ <tr>
193
+ <td>facebook/m2m100_418M</td>
194
+ <td>en-zh</td>
195
+ <td><a href="https://huggingface.co/facebook/m2m100_418M" target="_blank">open</a></td>
196
+ <td>461M</td>
197
+ <td>1,8729</td>
198
+ <td>23,8577</td>
199
+ <td>0,7829</td>
200
+ <td>121,8503</td>
201
+ <td>0,0799</td>
202
+ <td>0,2400</td>
203
+ <td>0,1111</td>
204
+ <td>0,2355</td>
205
+ <td>0,2354</td>
206
+ <td>1956,3508</td>
207
+ <td>408,8626</td>
208
+ </tr>
209
+ <tr>
210
+ <td>facebook/mbart-large-50-many-to-many-mmt</td>
211
+ <td>en-zh</td>
212
+ <td><a href="https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt" target="_blank">open</a></td>
213
+ <td>582M</td>
214
+ <td>1,7330</td>
215
+ <td>25,4084</td>
216
+ <td>0,8042</td>
217
+ <td><span style="font-weight: bold;background-color: Lime;">159,9984</td>
218
+ <td>0,0760</td>
219
+ <td>0,2178</td>
220
+ <td>0,1005</td>
221
+ <td>0,2162</td>
222
+ <td>0,2158</td>
223
+ <td>2539,4210</td>
224
+ <td>475,5201</td>
225
+ </tr>
226
+ <tr>
227
+ <td>facebook/nllb-200-distilled-1.3B</td>
228
+ <td>en-zh</td>
229
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-1.3B" target="_blank">open</a></td>
230
+ <td>1307M</td>
231
+ <td>1,2399</td>
232
+ <td>19,4471</td>
233
+ <td>0,7733</td>
234
+ <td>128,1332</td>
235
+ <td>0,0165</td>
236
+ <td>0,1706</td>
237
+ <td>0,0631</td>
238
+ <td>0,1690</td>
239
+ <td>0,1690</td>
240
+ <td>23758,4120</td>
241
+ <td>551,0516</td>
242
+ </tr>
243
+ <tr>
244
+ <td>facebook/nllb-200-distilled-600M</td>
245
+ <td>en-zh</td>
246
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-600M" target="_blank">open</a></td>
247
+ <td>586M</td>
248
+ <td>1,2022</td>
249
+ <td>18,8796</td>
250
+ <td>0,7673</td>
251
+ <td>113,6864</td>
252
+ <td>0,0162</td>
253
+ <td>0,1781</td>
254
+ <td>0,0648</td>
255
+ <td>0,1764</td>
256
+ <td>0,1763</td>
257
+ <td>1665,9829</td>
258
+ <td>403,9806</td>
259
+ </tr>
260
+ <tr>
261
+ <td>utrobinmv/t5_translate_en_ru_zh_base_200</td>
262
+ <td>en-zh</td>
263
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_base_200" target="_blank">open</a></span></td>
264
+ <td>284M</td>
265
+ <td><span style="font-weight: bold;background-color: Lime;">2,1928</td>
266
+ <td>29,1543</td>
267
+ <td>0,8131</td>
268
+ <td>113,8981</td>
269
+ <td>0,0830</td>
270
+ <td>0,2508</td>
271
+ <td>0,1173</td>
272
+ <td>0,2471</td>
273
+ <td>0,2474</td>
274
+ <td>1109,7135</td>
275
+ <td>365,6705</td>
276
+ </tr>
277
+ <tr>
278
+ <td>utrobinmv/t5_translate_en_ru_zh_large_1024</td>
279
+ <td>en-zh</td>
280
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024" target="_blank">open</a></span></td>
281
+ <td>811M</td>
282
+ <td>1,6960</td>
283
+ <td><span style="font-weight: bold;background-color: Lime;">29,6255</td>
284
+ <td><span style="font-weight: bold;background-color: Lime;">0,8265</td>
285
+ <td>115,7029</td>
286
+ <td><span style="font-weight: bold;background-color: Lime;">0,0869</td>
287
+ <td><span style="font-weight: bold;background-color: Lime;">0,2523</td>
288
+ <td><span style="font-weight: bold;background-color: Lime;">0,1197</td>
289
+ <td><span style="font-weight: bold;background-color: Lime;">0,2480</td>
290
+ <td><span style="font-weight: bold;background-color: Lime;">0,2480</td>
291
+ <td>46785,9260</td>
292
+ <td>774,6595</td>
293
+ </tr>
294
+ <tr>
295
+ <td>utrobinmv/t5_translate_en_ru_zh_small_1024</td>
296
+ <td>en-zh</td>
297
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_small_1024" target="_blank">open</a></span></td>
298
+ <td>105M</td>
299
+ <td>1,6180</td>
300
+ <td>22,6094</td>
301
+ <td>0,7773</td>
302
+ <td>116,6643</td>
303
+ <td>0,0726</td>
304
+ <td>0,2433</td>
305
+ <td>0,0964</td>
306
+ <td>0,2387</td>
307
+ <td>0,2390</td>
308
+ <td>404,2018</td>
309
+ <td>237,6188</td>
310
+ </tr>
311
+ <tr>
312
+ <td>Helsinki-NLP/opus-mt-ru-en</td>
313
+ <td>ru-en</td>
314
+ <td><a href="https://huggingface.co/Helsinki-NLP/opus-mt-ru-en" target="_blank">open</a></td>
315
+ <td>73M</td>
316
+ <td>23,9833</td>
317
+ <td>53,8174</td>
318
+ <td>0,9374</td>
319
+ <td>66,4136</td>
320
+ <td>0,5700</td>
321
+ <td>0,5747</td>
322
+ <td>0,3152</td>
323
+ <td>0,5222</td>
324
+ <td>0,5220</td>
325
+ <td>590,5894</td>
326
+ <td>184,9895</td>
327
+ </tr>
328
+ <tr>
329
+ <td>facebook/m2m100_418M</td>
330
+ <td>ru-en</td>
331
+ <td><a href="https://huggingface.co/facebook/m2m100_418M" target="_blank">open</a></td>
332
+ <td>461M</td>
333
+ <td>13,7000</td>
334
+ <td>43,5764</td>
335
+ <td>0,8976</td>
336
+ <td><span style="font-weight: bold;background-color: Lime;">82,6220</td>
337
+ <td>0,4345</td>
338
+ <td>0,4739</td>
339
+ <td>0,2104</td>
340
+ <td>0,4201</td>
341
+ <td>0,4198</td>
342
+ <td>2503,8058</td>
343
+ <td>524,1630</td>
344
+ </tr>
345
+ <tr>
346
+ <td>facebook/mbart-large-50-many-to-many-mmt</td>
347
+ <td>ru-en</td>
348
+ <td><a href="https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt" target="_blank">open</a></td>
349
+ <td>582M</td>
350
+ <td>24,3928</td>
351
+ <td>53,7343</td>
352
+ <td>0,9361</td>
353
+ <td>65,8848</td>
354
+ <td>0,5555</td>
355
+ <td>0,5669</td>
356
+ <td>0,3122</td>
357
+ <td>0,5177</td>
358
+ <td>0,5177</td>
359
+ <td>3452,1722</td>
360
+ <td>511,7801</td>
361
+ </tr>
362
+ <tr>
363
+ <td>facebook/nllb-200-distilled-1.3B</td>
364
+ <td>ru-en</td>
365
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-1.3B" target="_blank">open</a></td>
366
+ <td>1307M</td>
367
+ <td>25,1481</td>
368
+ <td>55,0731</td>
369
+ <td>0,9389</td>
370
+ <td>65,8016</td>
371
+ <td>0,5737</td>
372
+ <td>0,5878</td>
373
+ <td>0,3408</td>
374
+ <td>0,5367</td>
375
+ <td>0,5367</td>
376
+ <td>34099,3069</td>
377
+ <td>693,8489</td>
378
+ </tr>
379
+ <tr>
380
+ <td>facebook/nllb-200-distilled-600M</td>
381
+ <td>ru-en</td>
382
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-600M" target="_blank">open</a></td>
383
+ <td>586M</td>
384
+ <td>22,4326</td>
385
+ <td>52,0603</td>
386
+ <td>0,9285</td>
387
+ <td>70,5321</td>
388
+ <td>0,5291</td>
389
+ <td>0,5482</td>
390
+ <td>0,2938</td>
391
+ <td>0,4964</td>
392
+ <td>0,4961</td>
393
+ <td>1987,7717</td>
394
+ <td>474,5719</td>
395
+ </tr>
396
+ <tr>
397
+ <td>utrobinmv/t5_translate_en_ru_zh_base_200</td>
398
+ <td>ru-en</td>
399
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_base_200" target="_blank">open</a></span></td>
400
+ <td>284M</td>
401
+ <td><span style="font-weight: bold;background-color: Lime;">27,3035</td>
402
+ <td>55,9527</td>
403
+ <td>0,9407</td>
404
+ <td>63,3606</td>
405
+ <td>0,5918</td>
406
+ <td>0,5949</td>
407
+ <td>0,3409</td>
408
+ <td>0,5417</td>
409
+ <td>0,5418</td>
410
+ <td>1288,3096</td>
411
+ <td>516,6495</td>
412
+ </tr>
413
+ <tr>
414
+ <td>utrobinmv/t5_translate_en_ru_zh_large_1024</td>
415
+ <td>ru-en</td>
416
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024" target="_blank">open</a></span></td>
417
+ <td>811M</td>
418
+ <td>27,1314</td>
419
+ <td><span style="font-weight: bold;background-color: Lime;">56,2669</td>
420
+ <td><span style="font-weight: bold;background-color: Lime;">0,9429</td>
421
+ <td>63,9991</td>
422
+ <td><span style="font-weight: bold;background-color: Lime;">0,6043</td>
423
+ <td><span style="font-weight: bold;background-color: Lime;">0,6031</td>
424
+ <td><span style="font-weight: bold;background-color: Lime;">0,3538</td>
425
+ <td><span style="font-weight: bold;background-color: Lime;">0,5513</td>
426
+ <td><span style="font-weight: bold;background-color: Lime;">0,5513</td>
427
+ <td>63181,3672</td>
428
+ <td>1070,1940</td>
429
+ </tr>
430
+ <tr>
431
+ <td>utrobinmv/t5_translate_en_ru_zh_small_1024</td>
432
+ <td>ru-en</td>
433
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_small_1024" target="_blank">open</a></span></td>
434
+ <td>105M</td>
435
+ <td>19,9053</td>
436
+ <td>49,8936</td>
437
+ <td>0,9219</td>
438
+ <td>71,3320</td>
439
+ <td>0,5206</td>
440
+ <td>0,5303</td>
441
+ <td>0,2615</td>
442
+ <td>0,4731</td>
443
+ <td>0,4732</td>
444
+ <td>816,4949</td>
445
+ <td>366,7248</td>
446
+ </tr>
447
+ <tr>
448
+ <td>facebook/m2m100_418M</td>
449
+ <td>ru-zh</td>
450
+ <td><a href="https://huggingface.co/facebook/m2m100_418M" target="_blank">open</a></td>
451
+ <td>461M</td>
452
+ <td>0,9265</td>
453
+ <td>17,3133</td>
454
+ <td>0,7453</td>
455
+ <td>123,8558</td>
456
+ <td>0,0638</td>
457
+ <td>0,1948</td>
458
+ <td>0,0741</td>
459
+ <td>0,1915</td>
460
+ <td>0,1915</td>
461
+ <td>2608,6762</td>
462
+ <td>474,9864</td>
463
+ </tr>
464
+ <tr>
465
+ <td>facebook/mbart-large-50-many-to-many-mmt</td>
466
+ <td>ru-zh</td>
467
+ <td><a href="https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt" target="_blank">open</a></td>
468
+ <td>582M</td>
469
+ <td>0,3101</td>
470
+ <td>4,3907</td>
471
+ <td>0,5623</td>
472
+ <td><span style="font-weight: bold;background-color: Lime;">1407,8223</td>
473
+ <td>0,0506</td>
474
+ <td>0,0635</td>
475
+ <td>0,0202</td>
476
+ <td>0,0612</td>
477
+ <td>0,0613</td>
478
+ <td>2858,9354</td>
479
+ <td>515,5096</td>
480
+ </tr>
481
+ <tr>
482
+ <td>facebook/nllb-200-distilled-1.3B</td>
483
+ <td>ru-zh</td>
484
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-1.3B" target="_blank">open</a></td>
485
+ <td>1307M</td>
486
+ <td>0,9701</td>
487
+ <td>17,0343</td>
488
+ <td>0,7562</td>
489
+ <td>136,8758</td>
490
+ <td>0,0126</td>
491
+ <td>0,1795</td>
492
+ <td>0,0671</td>
493
+ <td>0,1770</td>
494
+ <td>0,1770</td>
495
+ <td>32840,6053</td>
496
+ <td>650,9642</td>
497
+ </tr>
498
+ <tr>
499
+ <td>facebook/nllb-200-distilled-600M</td>
500
+ <td>ru-zh</td>
501
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-600M" target="_blank">open</a></td>
502
+ <td>586M</td>
503
+ <td>0,8562</td>
504
+ <td>14,8349</td>
505
+ <td>0,7371</td>
506
+ <td>118,6932</td>
507
+ <td>0,0113</td>
508
+ <td>0,1631</td>
509
+ <td>0,0551</td>
510
+ <td>0,1604</td>
511
+ <td>0,1606</td>
512
+ <td>1477,4689</td>
513
+ <td>372,8740</td>
514
+ </tr>
515
+ <tr>
516
+ <td>joefox/mbart-large-ru-zh-ru-many-to-many-mmt</td>
517
+ <td>ru-zh</td>
518
+ <td><a href="https://huggingface.co/joefox/mbart-large-ru-zh-ru-many-to-many-mmt" target="_blank">open</a></td>
519
+ <td>382M</td>
520
+ <td>1,2970</td>
521
+ <td>22,5037</td>
522
+ <td>0,7852</td>
523
+ <td>113,3990</td>
524
+ <td>0,0726</td>
525
+ <td>0,2229</td>
526
+ <td>0,0928</td>
527
+ <td>0,2188</td>
528
+ <td>0,2192</td>
529
+ <td>1413,1433</td>
530
+ <td>329,3788</td>
531
+ </tr>
532
+ <tr>
533
+ <td>utrobinmv/t5_translate_en_ru_zh_base_200</td>
534
+ <td>ru-zh</td>
535
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_base_200" target="_blank">open</a></span></td>
536
+ <td>284M</td>
537
+ <td><span style="font-weight: bold;background-color: Lime;">1,5048</td>
538
+ <td>22,9738</td>
539
+ <td>0,7856</td>
540
+ <td>113,6822</td>
541
+ <td>0,0723</td>
542
+ <td>0,2256</td>
543
+ <td>0,0955</td>
544
+ <td>0,2217</td>
545
+ <td>0,2215</td>
546
+ <td>984,3104</td>
547
+ <td>395,4513</td>
548
+ </tr>
549
+ <tr>
550
+ <td>utrobinmv/t5_translate_en_ru_zh_large_1024</td>
551
+ <td>ru-zh</td>
552
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024" target="_blank">open</a></span></td>
553
+ <td>811M</td>
554
+ <td>1,2027</td>
555
+ <td><span style="font-weight: bold;background-color: Lime;">23,4712</td>
556
+ <td><span style="font-weight: bold;background-color: Lime;">0,7933</td>
557
+ <td>117,2472</td>
558
+ <td><span style="font-weight: bold;background-color: Lime;">0,0740</td>
559
+ <td><span style="font-weight: bold;background-color: Lime;">0,2267</td>
560
+ <td><span style="font-weight: bold;background-color: Lime;">0,0994</td>
561
+ <td><span style="font-weight: bold;background-color: Lime;">0,2233</td>
562
+ <td><span style="font-weight: bold;background-color: Lime;">0,2233</td>
563
+ <td>54047,8655</td>
564
+ <td>855,6965</td>
565
+ </tr>
566
+ <tr>
567
+ <td>utrobinmv/t5_translate_en_ru_zh_small_1024</td>
568
+ <td>ru-zh</td>
569
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_small_1024" target="_blank">open</a></span></td>
570
+ <td>105M</td>
571
+ <td>1,1654</td>
572
+ <td>17,3986</td>
573
+ <td>0,7525</td>
574
+ <td>112,1590</td>
575
+ <td>0,0634</td>
576
+ <td>0,2011</td>
577
+ <td>0,0730</td>
578
+ <td>0,1984</td>
579
+ <td>0,1986</td>
580
+ <td>503,1929</td>
581
+ <td>266,5707</td>
582
+ </tr>
583
+ <tr>
584
+ <td>Helsinki-NLP/opus-mt-zh-en</td>
585
+ <td>zh-en</td>
586
+ <td><a href="https://huggingface.co/Helsinki-NLP/opus-mt-zh-en" target="_blank">open</a></td>
587
+ <td>74M</td>
588
+ <td>17,8786</td>
589
+ <td>49,8389</td>
590
+ <td>0,9247</td>
591
+ <td>76,6661</td>
592
+ <td>0,5134</td>
593
+ <td>0,5292</td>
594
+ <td>0,2652</td>
595
+ <td>0,4525</td>
596
+ <td>0,4525</td>
597
+ <td>493,6717</td>
598
+ <td>194,4169</td>
599
+ </tr>
600
+ <tr>
601
+ <td>facebook/m2m100_418M</td>
602
+ <td>zh-en</td>
603
+ <td><a href="https://huggingface.co/facebook/m2m100_418M" target="_blank">open</a></td>
604
+ <td>461M</td>
605
+ <td>13,8451</td>
606
+ <td>45,7770</td>
607
+ <td>0,9141</td>
608
+ <td>80,4613</td>
609
+ <td>0,4578</td>
610
+ <td>0,4933</td>
611
+ <td>0,2226</td>
612
+ <td>0,4117</td>
613
+ <td>0,4115</td>
614
+ <td>2261,8730</td>
615
+ <td>467,0410</td>
616
+ </tr>
617
+ <tr>
618
+ <td>facebook/mbart-large-50-many-to-many-mmt</td>
619
+ <td>zh-en</td>
620
+ <td><a href="https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt" target="_blank">open</a></td>
621
+ <td>582M</td>
622
+ <td>20,0748</td>
623
+ <td>50,3136</td>
624
+ <td>0,9276</td>
625
+ <td>72,2366</td>
626
+ <td>0,5233</td>
627
+ <td>0,5399</td>
628
+ <td>0,2757</td>
629
+ <td>0,4615</td>
630
+ <td>0,4614</td>
631
+ <td>2748,5020</td>
632
+ <td>471,4562</td>
633
+ </tr>
634
+ <tr>
635
+ <td>facebook/nllb-200-distilled-1.3B</td>
636
+ <td>zh-en</td>
637
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-1.3B" target="_blank">open</a></td>
638
+ <td>1307M</td>
639
+ <td>13,3269</td>
640
+ <td>41,2097</td>
641
+ <td>0,8985</td>
642
+ <td><span style="font-weight: bold;background-color: Lime;">81,6055</td>
643
+ <td>0,4142</td>
644
+ <td>0,4491</td>
645
+ <td>0,2098</td>
646
+ <td>0,3790</td>
647
+ <td>0,3790</td>
648
+ <td>28471,2663</td>
649
+ <td>606,2567</td>
650
+ </tr>
651
+ <tr>
652
+ <td>facebook/nllb-200-distilled-600M</td>
653
+ <td>zh-en</td>
654
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-600M" target="_blank">open</a></td>
655
+ <td>586M</td>
656
+ <td>15,2232</td>
657
+ <td>44,9461</td>
658
+ <td>0,9112</td>
659
+ <td>81,2294</td>
660
+ <td>0,4532</td>
661
+ <td>0,4814</td>
662
+ <td>0,2248</td>
663
+ <td>0,4037</td>
664
+ <td>0,4037</td>
665
+ <td>1965,1474</td>
666
+ <td>420,0157</td>
667
+ </tr>
668
+ <tr>
669
+ <td>utrobinmv/t5_translate_en_ru_zh_base_200</td>
670
+ <td>zh-en</td>
671
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_base_200" target="_blank">open</a></span></td>
672
+ <td>284M</td>
673
+ <td>22,1332</td>
674
+ <td>52,4549</td>
675
+ <td>0,9315</td>
676
+ <td>70,2528</td>
677
+ <td>0,5579</td>
678
+ <td>0,5621</td>
679
+ <td>0,2959</td>
680
+ <td>0,4793</td>
681
+ <td>0,4795</td>
682
+ <td>1224,1559</td>
683
+ <td>505,7730</td>
684
+ </tr>
685
+ <tr>
686
+ <td>utrobinmv/t5_translate_en_ru_zh_large_1024</td>
687
+ <td>zh-en</td>
688
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024" target="_blank">open</a></span></td>
689
+ <td>811M</td>
690
+ <td><span style="font-weight: bold;background-color: Lime;">22,5254</td>
691
+ <td><span style="font-weight: bold;background-color: Lime;">53,2576</td>
692
+ <td><span style="font-weight: bold;background-color: Lime;">0,9361</td>
693
+ <td>69,8088</td>
694
+ <td><span style="font-weight: bold;background-color: Lime;">0,5741</td>
695
+ <td><span style="font-weight: bold;background-color: Lime;">0,5742</td>
696
+ <td><span style="font-weight: bold;background-color: Lime;">0,3118</td>
697
+ <td><span style="font-weight: bold;background-color: Lime;">0,4922</td>
698
+ <td><span style="font-weight: bold;background-color: Lime;">0,4922</td>
699
+ <td>57748,4735</td>
700
+ <td>978,7451</td>
701
+ </tr>
702
+ <tr>
703
+ <td>utrobinmv/t5_translate_en_ru_zh_small_1024</td>
704
+ <td>zh-en</td>
705
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_small_1024" target="_blank">open</a></span></td>
706
+ <td>105M</td>
707
+ <td>14,9457</td>
708
+ <td>45,8742</td>
709
+ <td>0,9132</td>
710
+ <td>78,4006</td>
711
+ <td>0,4781</td>
712
+ <td>0,4916</td>
713
+ <td>0,2148</td>
714
+ <td>0,4074</td>
715
+ <td>0,4074</td>
716
+ <td>554,3235</td>
717
+ <td>334,6551</td>
718
+ </tr>
719
+ <tr>
720
+ <td>facebook/m2m100_418M</td>
721
+ <td>zh-ru</td>
722
+ <td><a href="https://huggingface.co/facebook/m2m100_418M" target="_blank">open</a></td>
723
+ <td>461M</td>
724
+ <td>10,4357</td>
725
+ <td>38,6650</td>
726
+ <td>0,7828</td>
727
+ <td>87,6799</td>
728
+ <td>0,3206</td>
729
+ <td>0,2036</td>
730
+ <td>0,0708</td>
731
+ <td>0,1999</td>
732
+ <td>0,1997</td>
733
+ <td>2494,7562</td>
734
+ <td>516,2932</td>
735
+ </tr>
736
+ <tr>
737
+ <td>facebook/mbart-large-50-many-to-many-mmt</td>
738
+ <td>zh-ru</td>
739
+ <td><a href="https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt" target="_blank">open</a></td>
740
+ <td>582M</td>
741
+ <td>2,1387</td>
742
+ <td>6,6085</td>
743
+ <td>0,7093</td>
744
+ <td><span style="font-weight: bold;background-color: Lime;">108,0543</td>
745
+ <td>0,0948</td>
746
+ <td>0,0609</td>
747
+ <td>0,0158</td>
748
+ <td>0,0590</td>
749
+ <td>0,0595</td>
750
+ <td>2569,8568</td>
751
+ <td>463,5049</td>
752
+ </tr>
753
+ <tr>
754
+ <td>facebook/nllb-200-distilled-1.3B</td>
755
+ <td>zh-ru</td>
756
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-1.3B" target="_blank">open</a></td>
757
+ <td>1307M</td>
758
+ <td>8,9797</td>
759
+ <td>32,8740</td>
760
+ <td>0,7646</td>
761
+ <td>89,8886</td>
762
+ <td>0,2619</td>
763
+ <td>0,1919</td>
764
+ <td>0,0615</td>
765
+ <td>0,1877</td>
766
+ <td>0,1877</td>
767
+ <td>43398,0084</td>
768
+ <td>1090,9077</td>
769
+ </tr>
770
+ <tr>
771
+ <td>facebook/nllb-200-distilled-600M</td>
772
+ <td>zh-ru</td>
773
+ <td><a href="https://huggingface.co/facebook/nllb-200-distilled-600M" target="_blank">open</a></td>
774
+ <td>586M</td>
775
+ <td>8,6919</td>
776
+ <td>34,9578</td>
777
+ <td>0,7656</td>
778
+ <td>91,4727</td>
779
+ <td>0,2692</td>
780
+ <td>0,1957</td>
781
+ <td>0,0666</td>
782
+ <td>0,1926</td>
783
+ <td>0,1927</td>
784
+ <td>3392,8836</td>
785
+ <td>751,6092</td>
786
+ </tr>
787
+ <tr>
788
+ <td>joefox/mbart-large-ru-zh-ru-many-to-many-mmt</td>
789
+ <td>zh-ru</td>
790
+ <td><a href="https://huggingface.co/joefox/mbart-large-ru-zh-ru-many-to-many-mmt" target="_blank">open</a></td>
791
+ <td>382M</td>
792
+ <td><span style="font-weight: bold;background-color: Lime;">13,7238</td>
793
+ <td>41,5467</td>
794
+ <td>0,7985</td>
795
+ <td>83,3914</td>
796
+ <td><span style="font-weight: bold;background-color: Lime;">0,3652</td>
797
+ <td><span style="font-weight: bold;background-color: Lime;">0,2274</td>
798
+ <td>0,0811</td>
799
+ <td><span style="font-weight: bold;background-color: Lime;">0,2226</td>
800
+ <td><span style="font-weight: bold;background-color: Lime;">0,2228</td>
801
+ <td>1722,9847</td>
802
+ <td>390,9610</td>
803
+ </tr>
804
+ <tr>
805
+ <td>utrobinmv/t5_translate_en_ru_zh_base_200</td>
806
+ <td>zh-ru</td>
807
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_base_200" target="_blank">open</a></span></td>
808
+ <td>284M</td>
809
+ <td>13,0804</td>
810
+ <td>42,0661</td>
811
+ <td>0,8023</td>
812
+ <td>84,9454</td>
813
+ <td>0,3460</td>
814
+ <td>0,2217</td>
815
+ <td>0,0747</td>
816
+ <td>0,2169</td>
817
+ <td>0,2168</td>
818
+ <td>1297,4867</td>
819
+ <td>524,5561</td>
820
+ </tr>
821
+ <tr>
822
+ <td>utrobinmv/t5_translate_en_ru_zh_large_1024</td>
823
+ <td>zh-ru</td>
824
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024" target="_blank">open</a></span></td>
825
+ <td>811M</td>
826
+ <td>13,2207</td>
827
+ <td><span style="font-weight: bold;background-color: Lime;">42,4253</td>
828
+ <td><span style="font-weight: bold;background-color: Lime;">0,8077</td>
829
+ <td>97,7217</td>
830
+ <td>0,3575</td>
831
+ <td>0,2260</td>
832
+ <td><span style="font-weight: bold;background-color: Lime;">0,0843</td>
833
+ <td>0,2215</td>
834
+ <td>0,2215</td>
835
+ <td>62159,5527</td>
836
+ <td>1053,2818</td>
837
+ </tr>
838
+ <tr>
839
+ <td>utrobinmv/t5_translate_en_ru_zh_small_1024</td>
840
+ <td>zh-ru</td>
841
+ <td><span style="background-color: yellow;"><a href="https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_small_1024" target="_blank">open</a></span></td>
842
+ <td>105M</td>
843
+ <td>8,2438</td>
844
+ <td>35,7662</td>
845
+ <td>0,7711</td>
846
+ <td>90,7159</td>
847
+ <td>0,2737</td>
848
+ <td>0,2043</td>
849
+ <td>0,0667</td>
850
+ <td>0,1998</td>
851
+ <td>0,1999</td>
852
+ <td>835,4988</td>
853
+ <td>372,2559</td>
854
+ </tr>
855
+ </tbody>
856
+ </table>