mistrAND-v2-mistral-andalusian

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1894

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1
• training_steps: 650

Training results

Training Loss	Epoch	Step	Validation Loss
1.8749	0.0763	50	2.0418
1.5846	0.1527	100	1.7360
1.3465	0.2290	150	1.6464
1.5093	0.3053	200	1.5062
1.204	0.3817	250	1.4444
1.2046	0.4580	300	1.3813
1.2638	0.5344	350	1.3477
1.1846	0.6107	400	1.3114
1.2367	0.6870	450	1.2750
1.2948	0.7634	500	1.2401
1.3076	0.8397	550	1.2119
0.9692	0.9160	600	1.1995
1.1368	0.9924	650	1.1894

Framework versions

• PEFT 0.10.1.dev0
• Transformers 4.41.0.dev0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1