bart-cnn-summarizer-20k-3ep

This model is a fine-tuned version of facebook/bart-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4241

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
19.3029	0.16	100	3.7511
17.9130	0.32	200	3.6223
17.4411	0.48	300	3.6013
17.0836	0.64	400	3.5322
16.9614	0.8	500	3.5164
16.7463	0.96	600	3.4893
15.4827	1.12	700	3.5281
15.3151	1.28	800	3.4721
15.6138	1.44	900	3.4987
15.0429	1.6	1000	3.4431
15.4223	1.76	1100	3.4391
15.4545	1.92	1200	3.4466
14.5418	2.08	1300	3.4259
14.4296	2.24	1400	3.4293
14.4877	2.4	1500	3.4153
14.4625	2.56	1600	3.4258
14.1427	2.7200	1700	3.4220
14.4386	2.88	1800	3.4241

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2