MuseMistral-v3-35k
This model is a fine-tuned version of mistralai/Mistral-7B-v0.1
Model description
This model generates classical music in ABC notation with only 7B parameters and internet data related to ABC notation.
Training procedure
Trained on free GPUs provided by Kaggle using Axolotl.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 8
- seed: 3407
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10
- training_steps: 1000
- mixed_precision_training: Native AMP
Framework versions
- PEFT 0.7.1
- Transformers 4.36.0
- Pytorch 2.1.2
- Datasets 2.16.0
- Tokenizers 0.15.0
- Downloads last month
- 2
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no pipeline_tag.
Model tree for oof-baroomf/MuseMistral-v3-35k
Base model
mistralai/Mistral-7B-v0.1