Edit model card

mt5-small-finetuned-genius

This model is a fine-tuned version of google/mt5-small on the Genius Music dataset found here. The song lyrics and song titles were preprocessed and used for fine-tuning.

You can view more examples of this model's inference on the following Space.

Model description

Please visit: google/mt5-small

Intended uses & limitations

  • Intended Uses: Given song lyrics, generate a summary.
  • Limitations: Due to the nature of music, the model can generate summaries containing hate speech.

Training and evaluation data

  • 27.6K Training Samples
  • 3.45 Validation Samples

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 4e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
7.9304 1.0 863 3.5226 14.235 6.78 14.206 14.168
3.8394 2.0 1726 3.0382 22.97 13.166 22.981 22.944
3.3799 3.0 2589 2.9010 24.932 14.54 24.929 24.919
3.2204 4.0 3452 2.8441 26.678 15.587 26.624 26.665
3.1498 5.0 4315 2.8363 26.827 15.696 26.773 26.793

Framework versions

  • Transformers 4.31.0
  • Pytorch 2.0.1+cu117
  • Datasets 2.14.1
  • Tokenizers 0.13.3
Downloads last month
23

Finetuned from

Dataset used to train miscjose/mt5-small-finetuned-genius-music

Space using miscjose/mt5-small-finetuned-genius-music 1