ateneoscsl/BUOD_distilBART_TM

📋 BUOD: distilBART Transformer Model

Authors: James Esguerra, Julia Avila, Hazielle Bugayong

This model is a fine-tuned version of sshleifer/distilbart-cnn-12-6 on the KAMI-3000 dataset, for the task of Filipino Text Summarization.

It achieves the following results on the evaluation set:

• Loss: 1.8049
• Rouge1: 50.5143
• Rouge2: 23.2481
• Rougel: 34.135
• Rougelsum: 46.4261

🔧 Finetuning/ Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.1377	1.0	586	1.8792	49.8737	22.7881	33.6698	45.8037
1.5731	2.0	1172	1.8049	50.5143	23.2481	34.135	46.4261

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1+cu113
• Datasets 2.7.1
• Tokenizers 0.13.2