metadata
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- ccmatrix
metrics:
- bleu
model-index:
- name: t5-small-finetuned-en-to-it-b32
results:
- task:
name: Sequence-to-sequence Language Modeling
type: text2text-generation
dataset:
name: ccmatrix
type: ccmatrix
config: en-it
split: train[3000:12000]
args: en-it
metrics:
- name: Bleu
type: bleu
value: 9.6816
t5-small-finetuned-en-to-it-b32
This model is a fine-tuned version of t5-small on the ccmatrix dataset. It achieves the following results on the evaluation set:
- Loss: 2.1496
- Bleu: 9.6816
- Gen Len: 56.5347
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 40
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|---|---|---|---|---|---|
| No log | 1.0 | 282 | 2.9409 | 2.6764 | 69.2487 |
| 3.3809 | 2.0 | 564 | 2.8277 | 2.4974 | 87.428 |
| 3.3809 | 3.0 | 846 | 2.7483 | 2.6851 | 89.7887 |
| 3.1255 | 4.0 | 1128 | 2.6831 | 3.1801 | 85.6927 |
| 3.1255 | 5.0 | 1410 | 2.6293 | 3.6949 | 79.9467 |
| 2.9965 | 6.0 | 1692 | 2.5809 | 4.0149 | 76.852 |
| 2.9965 | 7.0 | 1974 | 2.5403 | 4.3463 | 74.6487 |
| 2.9002 | 8.0 | 2256 | 2.5033 | 4.838 | 72.6053 |
| 2.8229 | 9.0 | 2538 | 2.4694 | 5.2829 | 67.984 |
| 2.8229 | 10.0 | 2820 | 2.4421 | 5.4964 | 68.986 |
| 2.76 | 11.0 | 3102 | 2.4135 | 5.8118 | 66.528 |
| 2.76 | 12.0 | 3384 | 2.3897 | 6.1966 | 65.052 |
| 2.7051 | 13.0 | 3666 | 2.3667 | 6.452 | 64.2273 |
| 2.7051 | 14.0 | 3948 | 2.3465 | 6.6428 | 63.516 |
| 2.6568 | 15.0 | 4230 | 2.3265 | 6.9467 | 61.8673 |
| 2.6183 | 16.0 | 4512 | 2.3101 | 7.2029 | 60.7393 |
| 2.6183 | 17.0 | 4794 | 2.2954 | 7.4982 | 60.0327 |
| 2.5757 | 18.0 | 5076 | 2.2799 | 7.7555 | 59.968 |
| 2.5757 | 19.0 | 5358 | 2.2660 | 7.8406 | 60.0307 |
| 2.5534 | 20.0 | 5640 | 2.2558 | 8.0679 | 59.0793 |
| 2.5534 | 21.0 | 5922 | 2.2426 | 8.3325 | 58.5367 |
| 2.5159 | 22.0 | 6204 | 2.2324 | 8.3538 | 58.6893 |
| 2.5159 | 23.0 | 6486 | 2.2217 | 8.5867 | 57.7627 |
| 2.4983 | 24.0 | 6768 | 2.2135 | 8.8324 | 56.7367 |
| 2.4791 | 25.0 | 7050 | 2.2052 | 8.8113 | 57.4373 |
| 2.4791 | 26.0 | 7332 | 2.1981 | 9.0909 | 57.0173 |
| 2.4529 | 27.0 | 7614 | 2.1908 | 9.0056 | 57.802 |
| 2.4529 | 28.0 | 7896 | 2.1856 | 9.2696 | 56.9773 |
| 2.4395 | 29.0 | 8178 | 2.1780 | 9.2824 | 57.0007 |
| 2.4395 | 30.0 | 8460 | 2.1722 | 9.2106 | 56.9893 |
| 2.4277 | 31.0 | 8742 | 2.1685 | 9.4668 | 56.406 |
| 2.4181 | 32.0 | 9024 | 2.1646 | 9.4992 | 56.2327 |
| 2.4181 | 33.0 | 9306 | 2.1616 | 9.5054 | 56.3033 |
| 2.4071 | 34.0 | 9588 | 2.1578 | 9.5093 | 56.548 |
| 2.4071 | 35.0 | 9870 | 2.1554 | 9.5227 | 56.7807 |
| 2.3991 | 36.0 | 10152 | 2.1532 | 9.5762 | 56.756 |
| 2.3991 | 37.0 | 10434 | 2.1518 | 9.6659 | 56.5913 |
| 2.3955 | 38.0 | 10716 | 2.1506 | 9.7199 | 56.5753 |
| 2.3955 | 39.0 | 10998 | 2.1498 | 9.6715 | 56.558 |
| 2.3913 | 40.0 | 11280 | 2.1496 | 9.6816 | 56.5347 |
Framework versions
- Transformers 4.22.1
- Pytorch 1.12.1
- Datasets 2.5.1
- Tokenizers 0.11.0