File size: 3,709 Bytes
efdb08d b506a6c efdb08d b506a6c efdb08d d886260 99cbd07 efdb08d b506a6c efdb08d c52834b efdb08d c52834b efdb08d c52834b efdb08d c52834b efdb08d d886260 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
---
license: mit
tags:
- generated_from_trainer
datasets:
- cartesinus/iva_mt_wslot
metrics:
- bleu
model-index:
- name: iva_mt_wslot-m2m100_418M-en-de
results:
- task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation
dataset:
name: iva_mt_wslot
type: iva_mt_wslot
config: en-de
split: validation
args: en-de
metrics:
- name: Bleu
type: bleu
value: 66.5548
language:
- en
- de
pipeline_tag: translation
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# iva_mt_wslot-m2m100_418M-en-de
This model is a fine-tuned version of [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M) on the iva_mt_wslot dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0126
- Bleu: 66.5548
- Gen Len: 20.6835
## Model description
More information needed
## How to use
First please make sure to install `pip install transformers`. First download model:
```python
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
import torch
def translate(input_text, lang):
input_ids = tokenizer(input_text, return_tensors="pt")
generated_tokens = model.generate(**input_ids, forced_bos_token_id=tokenizer.get_lang_id(lang))
return tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
model_name = "cartesinus/iva_mt_wslot-m2m100_418M-0.1.0-en-de"
tokenizer = M2M100Tokenizer.from_pretrained(model_name, src_lang="en", tgt_lang="de")
model = M2M100ForConditionalGeneration.from_pretrained(model_name)
```
Then you can translate either plain text like this:
```python
print(translate("set the temperature on my thermostat", "de"))
```
or you can translate with slot annotations that will be restored in tgt language:
```python
print(translate("wake me up at <a>nine am<a> on <b>friday<b>", "de"))
```
Limitations of translation with slot transfer:
1) Annotated words must be placed between semi-xml tags like this "this is \<a\>example\<a\>"
2) There is no closing tag for example "\<\a\>" in the above example - this is done on purpose to omit problems with backslash escape
3) If the sentence consists of more than one slot then simply use the next alphabet letter. For example "this is \<a\>example\<a\> with more than \<b\>one\<b\> slot"
4) Please do not add space before the first or last annotated word because this particular model was trained this way and it most probably will lower its results
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 7
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|
| 0.0183 | 1.0 | 1884 | 0.0144 | 63.5045 | 20.2994 |
| 0.0119 | 2.0 | 3768 | 0.0127 | 66.0473 | 20.5423 |
| 0.0083 | 3.0 | 5652 | 0.0123 | 65.5139 | 20.5409 |
| 0.0065 | 4.0 | 7536 | 0.0124 | 66.0731 | 20.6114 |
| 0.0048 | 5.0 | 9420 | 0.0122 | 66.2454 | 20.5906 |
| 0.0038 | 6.0 | 11304 | 0.0124 | 66.7412 | 20.6263 |
| 0.0031 | 7.0 | 13188 | 0.0126 | 66.5548 | 20.6835 |
### Framework versions
- Transformers 4.28.1
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3 |