liputan6-base

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on the id_liputan6 canonical dataset. It achieves the following results on the evaluation set:

• Loss: 5.4266
• Rouge1: 18.1827
• Rouge2: 5.5014
• Rougel: 15.5147
• Rougelsum: 16.9245
• Gen Len: 35.116

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.8271	1.0	63	3.9787	14.5233	4.127	12.7611	13.5205	47.473
2.2739	2.0	126	4.1316	15.9563	4.7752	13.8242	14.8005	44.229
1.2999	3.0	189	4.4850	17.2932	4.6352	14.8582	16.1555	33.112
0.6423	4.0	252	4.9200	17.5707	4.9772	14.949	16.1838	36.399
0.2536	5.0	315	5.4266	17.698	4.7021	14.8138	16.3595	31.108

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1