MuseMistral-v3-35k

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1

Model description

This model generates classical music in ABC notation with only 7B parameters and internet data related to ABC notation.

Training procedure

Trained on free GPUs provided by Kaggle using Axolotl.

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 3407
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 10
  • training_steps: 1000
  • mixed_precision_training: Native AMP

Framework versions

  • PEFT 0.7.1
  • Transformers 4.36.0
  • Pytorch 2.1.2
  • Datasets 2.16.0
  • Tokenizers 0.15.0
Downloads last month
0
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for oof-baroomf/MuseMistral-v3-35k

Adapter
(1802)
this model