MiuN2k3/BARTpho-syllable-normalize-text

ViT5-base-normalized

This model is a fine-tuned version of vinai/bartpho-syllable on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0194
• Rouge1: 89.1883
• Rouge2: 85.545
• Rougel: 89.0077
• Rougelsum: 89.0313
• Gen Len: 15.0895

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.0911	1.0	524	0.0287	86.8798	81.8723	86.6699	86.7029	15.1629
0.0213	2.0	1048	0.0216	88.2321	84.1301	88.0463	88.0681	15.1238
0.0147	3.0	1572	0.0202	88.8138	84.9897	88.6248	88.6669	15.08
0.0115	4.0	2096	0.0203	89.1624	85.4043	88.9621	88.9905	15.0676
0.0094	5.0	2620	0.0194	89.1883	85.545	89.0077	89.0313	15.0895

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0
• Datasets 3.0.0
• Tokenizers 0.19.1