José Ángel González
Update README.md
5e6c676
metadata
license: apache-2.0
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: barthez-deft-linguistique
    results:
      - task:
          name: Summarization
          type: summarization
        metrics:
          - name: Rouge1
            type: rouge
            value: 41.989

barthez-deft-linguistique

This model is a fine-tuned version of moussaKam/barthez on an unknown dataset.

Note: this model is one of the preliminary experiments and it underperforms the models published in the paper (using MBartHez and HAL/Wiki pre-training + copy mechanisms)

It achieves the following results on the evaluation set:

  • Loss: 1.7596
  • Rouge1: 41.989
  • Rouge2: 22.4524
  • Rougel: 32.7966
  • Rougelsum: 32.7953
  • Gen Len: 22.1549

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
3.0569 1.0 108 2.0282 31.6993 14.9483 25.5565 25.4379 18.3803
2.2892 2.0 216 1.8553 35.2563 18.019 28.3135 28.2927 18.507
1.9062 3.0 324 1.7696 37.4613 18.1488 28.9959 29.0134 19.5352
1.716 4.0 432 1.7641 37.6903 18.7496 30.1097 30.1027 18.9577
1.5722 5.0 540 1.7781 38.1013 19.8291 29.8142 29.802 19.169
1.4655 6.0 648 1.7661 38.3557 20.3309 30.5068 30.4728 19.3662
1.3507 7.0 756 1.7596 39.7409 20.2998 31.0849 31.1152 19.3944
1.2874 8.0 864 1.7706 37.7846 20.3457 30.6826 30.6321 19.4789
1.2641 9.0 972 1.7848 38.7421 19.5701 30.5798 30.6305 19.3944
1.1192 10.0 1080 1.8008 40.3313 20.3378 31.8325 31.8648 19.5493
1.0724 11.0 1188 1.8450 38.9612 20.5719 31.4496 31.3144 19.8592
1.0077 12.0 1296 1.8364 36.5997 18.46 29.1808 29.1705 19.7324
0.9362 13.0 1404 1.8677 38.0371 19.2321 30.3893 30.3926 19.6338
0.8868 14.0 1512 1.9154 36.4737 18.5314 29.325 29.3634 19.6479
0.8335 15.0 1620 1.9344 35.7583 18.0687 27.9666 27.8675 19.8028
0.8305 16.0 1728 1.9556 37.2137 18.2199 29.5959 29.5799 19.9577
0.8057 17.0 1836 1.9793 36.6834 17.8505 28.6701 28.7145 19.7324
0.7869 18.0 1944 1.9994 37.5918 19.1984 28.8569 28.8278 19.7606
0.7549 19.0 2052 2.0117 37.3278 18.5169 28.778 28.7737 19.8028
0.7497 20.0 2160 2.0189 37.7513 19.1813 29.3675 29.402 19.6901

Framework versions

  • Transformers 4.10.2
  • Pytorch 1.7.1+cu110
  • Datasets 1.11.0
  • Tokenizers 0.10.3