---
license: mit
language:
- de
- en
- ja
- ru
- is
- cs
- zh
pipeline_tag: translation
---
facebook wmt21 model [facebook/wmt21-dense-24-wide-en-x](https://huggingface.co/facebook/wmt21-dense-24-wide-en-x/) as safetensors for testing. 

so far, looks quite good, cf. [comet](https://github.com/Unbabel/COMET) scores:

```code
+-----------------------------------------+-----------------+
| File                                    |   Overall Score |
+=========================================+=================+
| Capybara_de_wmt21_scored.jsonl          |        0.848375 |
+-----------------------------------------+-----------------+
| Capybara_de_GPT4_scored.jsonl           |        0.846241 |
+-----------------------------------------+-----------------+
| Capybara_de_Claude-3-Opus_scored.jsonl  |        0.84568  |
+-----------------------------------------+-----------------+
| Capybara_de_deepl_scored.jsonl          |        0.843937 |
+-----------------------------------------+-----------------+
| Capybara_de_GPT3.5_scored.jsonl         |        0.843922 |
+-----------------------------------------+-----------------+
| Capybara_de_occiglot_scored.jsonl       |        0.83135  |
+-----------------------------------------+-----------------+
| Capybara_de_discolm_scored.jsonl        |        0.830676 |
+-----------------------------------------+-----------------+
| Capybara_de_nbbl_scored.jsonl           |        0.829132 |
+-----------------------------------------+-----------------+
| Capybara_de_wmt19_scored.jsonl          |        0.824847 |
+-----------------------------------------+-----------------+
| Capybara_de_t5madlad_scored.jsonl       |        0.818146 |
+-----------------------------------------+-----------------+
| Capybara_de_mixtral_scored.jsonl        |        0.807397 |
+-----------------------------------------+-----------------+
| Capybara_de_TowerInstruct2_scored.jsonl |        0.788971 |
+-----------------------------------------+-----------------+
```

also, cf. comparison on a few snippets: https://huggingface.co/spaces/cstr/compare_translations

regarding quantization: on linux, or windows wsl (with accelerate, triton), you can use quantized versions 
[q8](https://huggingface.co/cstr/wmt21-dense-24-wide-en-x-stq8)
[q4](https://huggingface.co/cstr/wmt21-dense-24-wide-en-x-stq4)

on apple mac mps, you can use [CTranslate2](https://github.com/OpenNMT/CTranslate2) like this:
first convert the model:
```shell
ct2-transformers-converter --model cstr/wmt21-dense-24-wide-en-x-st --quantization int8_float32 --output_dir wmt21ct2_int8
```

then run e.g. in python:

```python
import ctranslate2
import transformers

translator = ctranslate2.Translator("wmt21ct2_int8")
tokenizer = transformers.AutoTokenizer.from_pretrained("facebook/wmt21-dense-24-wide-en-x")
tokenizer.src_lang = "en"

source = tokenizer.convert_ids_to_tokens(tokenizer.encode("Choose the correct verb form to complete the sentence: The birds ____________ (to fly) to the south for the winter."))
target_prefix = [tokenizer.lang_code_to_token["de"]]
results = translator.translate_batch([source], target_prefix=[target_prefix])
target = results[0].hypotheses[0][1:]

print(tokenizer.decode(tokenizer.convert_tokens_to_ids(target)))
```