liputan6-seq_bn-rf64

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on the id_liputan6 canonical dataset. It achieves the following results on the evaluation set:

• Loss: 2.7425
• Rouge1: 25.4752
• Rouge2: 11.3075
• Rougel: 21.8512
• Rougelsum: 23.007
• Gen Len: 41.252

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.7121	1.0	63	2.7515	24.5986	10.2547	20.7138	22.4882	33.171
2.3483	2.0	126	2.7425	25.6586	10.7898	21.7223	23.503	33.715
2.2546	3.0	189	2.6605	24.6676	10.1646	20.9376	22.7405	29.451
2.1725	4.0	252	2.7043	25.3421	10.5863	21.3692	23.2586	30.896
2.1135	5.0	315	2.7079	24.9923	10.432	21.1505	22.9477	32.451

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1