BUOD_distilBART_TM / README.md
0xhaz's picture
Create README.md
351ed70
metadata
license: apache-2.0
tags:
  - generated_from_trainer
metrics:
  - rouge
  - meteor
model-index:
  - name: distilbart-cnn-12-6-finetuned-1.3.2
    results: []
datasets:
  - ateneoscsl/BUOD_articlescraper
  - cnn_dailymail
  - xsum
language:
  - tl
  - en

πŸ“‹ BUOD: distilBART Transformer Model

Model:distilBART Authors: James Esguerra, Julia Avila, Hazielle Bugayong

This model is a fine-tuned version of sshleifer/distilbart-cnn-12-6 on the KAMI-3000 dataset, for the task of Filipino Text Summarization.

It achieves the following results on the evaluation set:

  • Loss: 1.8049
  • Rouge1: 50.5143
  • Rouge2: 23.2481
  • Rougel: 34.135
  • Rougelsum: 46.4261

πŸ”§ Finetuning/ Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 2

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
2.1377 1.0 586 1.8792 49.8737 22.7881 33.6698 45.8037
1.5731 2.0 1172 1.8049 50.5143 23.2481 34.135 46.4261

Framework versions

  • Transformers 4.25.1
  • Pytorch 1.12.1+cu113
  • Datasets 2.7.1
  • Tokenizers 0.13.2