hubert2BertMusicTestL5e7

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2639
• Rouge1: 28.296
• Rouge2: 7.1646
• Rougel: 20.2478
• Rougelsum: 20.191
• Gen Len: 58.92

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
4.7701	1.0	1361	3.6345	29.4772	6.5481	21.1446	21.1031	51.04
3.3104	2.0	2722	2.9389	24.7952	6.9239	19.4001	19.4318	31.01
2.9088	3.0	4083	2.6221	29.6422	7.2219	21.0592	21.0778	57.25
2.7274	4.0	5444	2.4599	28.0386	8.311	20.629	20.6316	50.06
2.6417	5.0	6805	2.3503	28.7788	8.8058	21.2309	21.2204	57.54
2.5002	6.0	8166	2.2905	31.5292	8.3235	22.2364	22.2584	65.09
2.4481	7.0	9527	2.2639	28.296	7.1646	20.2478	20.191	58.92

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.2
• Tokenizers 0.13.3