led-base-16384-text_summarization_data

This model is a fine-tuned version of allenai/led-base-16384 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9531
• Rouge1: 43.3689
• Rouge2: 19.9885
• RougeL: 39.9887
• RougeLsum: 40.0679
• Gen Len: 14.0392

Model description

This is a text summarization model.

For more information on how it was created, check out the following link: https://github.com/DunnBC22/NLP_Projects/blob/main/Text%20Summarization/Text-Summarized%20Data%20-%20Comparison/LED%20-%20Text%20Summarization%20-%204%20Epochs.ipynb

Intended uses & limitations

This model is intended to demonstrate my ability to solve a complex problem using technology.

Training and evaluation data

Dataset Source: https://www.kaggle.com/datasets/cuitengfeui/textsummarization-data

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	RougeL	RougeLsum	General Length
1.329	1.0	1197	0.9704	42.4111	19.8995	39.4717	39.5449	14.254
0.8367	2.0	2394	0.9425	43.1141	19.6089	39.7533	39.8298	14.1058
0.735	3.0	3591	0.9421	42.8101	19.8281	39.617	39.6751	13.7101
0.6737	4.0	4788	0.9531	43.3689	19.9885	39.9887	40.0679	14.0392

Framework versions

• Transformers 4.26.1
• Pytorch 1.12.1
• Datasets 2.9.0
• Tokenizers 0.12.1