mt5-small-finetuned-genius
This model is a fine-tuned version of google/mt5-small on the Genius Music dataset found here. The song lyrics and song titles were preprocessed and used for fine-tuning.
You can view more examples of this model's inference on the following Space.
Model description
Please visit: google/mt5-small
Intended uses & limitations
- Intended Uses: Given song lyrics, generate a summary.
- Limitations: Due to the nature of music, the model can generate summaries containing hate speech.
Training and evaluation data
- 27.6K Training Samples
- 3.45 Validation Samples
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 4e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|---|---|---|---|---|---|---|---|
| 7.9304 | 1.0 | 863 | 3.5226 | 14.235 | 6.78 | 14.206 | 14.168 |
| 3.8394 | 2.0 | 1726 | 3.0382 | 22.97 | 13.166 | 22.981 | 22.944 |
| 3.3799 | 3.0 | 2589 | 2.9010 | 24.932 | 14.54 | 24.929 | 24.919 |
| 3.2204 | 4.0 | 3452 | 2.8441 | 26.678 | 15.587 | 26.624 | 26.665 |
| 3.1498 | 5.0 | 4315 | 2.8363 | 26.827 | 15.696 | 26.773 | 26.793 |
Framework versions
- Transformers 4.31.0
- Pytorch 2.0.1+cu117
- Datasets 2.14.1
- Tokenizers 0.13.3
- Downloads last month
- 84
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for miscjose/mt5-small-finetuned-genius-music
Base model
google/mt5-small