oMateos2020's picture
update model card README.md
3b938dc
metadata
tags:
  - generated_from_trainer
datasets:
  - xsum
metrics:
  - rouge
model-index:
  - name: XSum_t5-small_800_adafactor
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: xsum
          type: xsum
          args: default
        metrics:
          - name: Rouge1
            type: rouge
            value: 33.022

XSum_t5-small_800_adafactor

This model is a fine-tuned version of /content/XSum_t5-small_800_adafactor/checkpoint-11000 on the xsum dataset. It achieves the following results on the evaluation set:

  • Loss: 2.1714
  • Rouge1: 33.022
  • Rouge2: 11.9979
  • Rougel: 26.7476
  • Rougelsum: 26.7402
  • Gen Len: 18.7543

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 25
  • eval_batch_size: 25
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 2
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
2.3404 0.01 100 2.2058 32.4826 11.5807 26.2716 26.2611 18.7842
2.3194 0.02 200 2.2028 32.6393 11.661 26.372 26.3643 18.788
2.3247 0.04 300 2.1999 32.6792 11.6985 26.3876 26.3786 18.7354
2.3276 0.05 400 2.1979 32.6668 11.7272 26.3964 26.3907 18.7957
2.317 0.06 500 2.1957 32.8267 11.8165 26.5075 26.4997 18.7543
2.3214 0.07 600 2.1942 32.8319 11.8064 26.5428 26.5448 18.7693
2.3014 0.09 700 2.1931 32.7136 11.7334 26.4958 26.486 18.7759
2.3294 0.1 800 2.1902 32.6818 11.7684 26.4314 26.4242 18.785
2.299 0.11 900 2.1914 32.672 11.7606 26.4475 26.4367 18.7853
2.3009 0.12 1000 2.1900 32.7816 11.7958 26.5167 26.5099 18.7685
2.2913 0.13 1100 2.1885 32.6438 11.7398 26.4077 26.4051 18.7742
2.293 0.15 1200 2.1854 32.8228 11.841 26.548 26.5415 18.7899
2.2857 0.16 1300 2.1853 32.7118 11.7439 26.4989 26.4941 18.7998
2.2921 0.17 1400 2.1832 32.6705 11.7333 26.4076 26.4082 18.8017
2.3074 0.18 1500 2.1827 32.7543 11.7787 26.4904 26.4923 18.7827
2.3044 0.2 1600 2.1806 32.8573 11.8672 26.5655 26.5619 18.8097
2.2922 0.21 1700 2.1819 32.8394 11.8158 26.5523 26.5467 18.7891
2.2901 0.22 1800 2.1803 32.7219 11.7493 26.4644 26.4572 18.7882
2.286 0.23 1900 2.1790 32.7474 11.852 26.5078 26.5014 18.7699
2.298 0.25 2000 2.1781 32.8662 11.8878 26.618 26.6174 18.7979
2.2787 0.26 2100 2.1775 32.9621 11.9521 26.6955 26.6914 18.7934
2.2823 0.27 2200 2.1777 33.0633 12.0622 26.7715 26.7597 18.7954
2.2889 0.28 2300 2.1742 32.9637 12.0154 26.6771 26.6721 18.7844
2.2847 0.29 2400 2.1774 32.7435 11.8869 26.5334 26.5306 18.756
2.2923 0.31 2500 2.1754 32.8437 11.8977 26.59 26.587 18.7964
2.2877 0.32 2600 2.1740 32.9137 11.9267 26.618 26.6046 18.7678
2.2976 0.33 2700 2.1728 32.9372 11.9048 26.6412 26.6345 18.7838
2.2935 0.34 2800 2.1719 32.7338 11.7836 26.5667 26.5629 18.7659
2.2622 0.36 2900 2.1718 32.9847 11.978 26.7093 26.7008 18.7627
2.2749 0.37 3000 2.1710 32.9835 11.9809 26.7034 26.6946 18.8016
2.2615 0.38 3100 2.1721 32.9343 11.9317 26.6752 26.6695 18.7689
2.2825 0.39 3200 2.1714 33.022 11.9979 26.7476 26.7402 18.7543

Framework versions

  • Transformers 4.20.1
  • Pytorch 1.12.0+cu113
  • Datasets 2.3.2
  • Tokenizers 0.12.1