Update README.md
Browse filesAdd Aya Expanse 8b and Hunyuan-MT metrics
README.md
CHANGED
|
@@ -106,11 +106,13 @@ The model is in `ctranslate2` format, and the tokenizers are `sentencepiece`, so
|
|
| 106 |
`bleu` and `chrf2` are calculated with [sacrebleu](https://github.com/mjpost/sacrebleu) on the [Flores200 `devtest` test set](https://huggingface.co/datasets/facebook/flores), `comet22` with the [`comet`](https://github.com/Unbabel/COMET) library and the [default model](https://huggingface.co/Unbabel/wmt22-comet-da). "Time (s)" is the time in seconds to translate the flores-devtest dataset (1012 sentences) on an RTX 4070s GPU with batch size 32.
|
| 107 |
|
| 108 |
|
| 109 |
-
|
|
| 110 |
-
|:---------------------------------|-------:|--------:|----------:|-----------:|
|
| 111 |
-
| quickmt/quickmt-ja-en
|
| 112 |
-
| Helsink-NLP/opus-mt-ja-en
|
| 113 |
-
|
|
| 114 |
-
| facebook/nllb-200-distilled-
|
| 115 |
-
| facebook/
|
| 116 |
-
| facebook/
|
|
|
|
|
|
|
|
|
| 106 |
`bleu` and `chrf2` are calculated with [sacrebleu](https://github.com/mjpost/sacrebleu) on the [Flores200 `devtest` test set](https://huggingface.co/datasets/facebook/flores), `comet22` with the [`comet`](https://github.com/Unbabel/COMET) library and the [default model](https://huggingface.co/Unbabel/wmt22-comet-da). "Time (s)" is the time in seconds to translate the flores-devtest dataset (1012 sentences) on an RTX 4070s GPU with batch size 32.
|
| 107 |
|
| 108 |
|
| 109 |
+
| | bleu | chrf2 | comet22 | Time (s) |
|
| 110 |
+
|:---------------------------------------|-------:|--------:|----------:|-----------:|
|
| 111 |
+
| quickmt/quickmt-ja-en | 28.87 | 58.56 | 87.83 | 1.10 |
|
| 112 |
+
| Helsink-NLP/opus-mt-ja-en | 19.22 | 49.15 | 82.64 | 3.54 |
|
| 113 |
+
| tencent/Hunyuan-MT-7B-fp8 | 22.79 | 56.73 | 87.94 | 22 |
|
| 114 |
+
| facebook/nllb-200-distilled-600M | 24.05 | 53.39 | 85.84 | 22.5 |
|
| 115 |
+
| facebook/nllb-200-distilled-1.3B | 28.4 | 56.96 | 87.47 | 37.15 |
|
| 116 |
+
| facebook/m2m100_418M | 18.82 | 49.55 | 82.59 | 18.27 |
|
| 117 |
+
| facebook/m2m100_1.2B | 23.32 | 53.46 | 85.43 | 35.44 |
|
| 118 |
+
| CohereLabs/aya-expanse-8b (bnb quant) | 26.40 | 56.92 | 87.97 | 73 |
|