Music Llama Medium

Llama for MIDI music generation

Model Details

Model Description

• Developed by: SkyTNT
• Trained by: asigalov61
• Model type: Llama Transformer
• License: apache-2.0

Model Sources

• Code Repository: https://github.com/SkyTNT/midi-model
• Demo: https://huggingface.co/spaces/skytnt/midi-composer

Training Details

Training Data

• projectlosangeles/Monster-MIDI-Dataset
• SymphonyNet MIDI Dataset

Training Hyperparameters

• config: tv2o-medium
• max-len: 4096
• layers: 12
• acc-grad: 8
• lr: 1e-4
• weight-decay: 0.01
• batch: 8x2x8
• bf16 mixed precision

Loss

• train loss: 0.2

Project Los Angeles

Tegridy Code 2024