nkerr/s2-3-mistral7b

s1-2-mistral7b

This model is a fine-tuned version of mistralai/Mistral-7B-v0.3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3101

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 9
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 36
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.0792	0.2094	100	0.3145
0.065	0.4188	200	0.3112
0.0629	0.6283	300	0.3048
0.061	0.8377	400	0.3044
0.0588	1.0461	500	0.3087
0.057	1.2555	600	0.3077
0.0569	1.4649	700	0.3057
0.0563	1.6743	800	0.3076
0.0557	1.8838	900	0.3066
0.0539	2.0921	1000	0.3074
0.0527	2.3016	1100	0.3088
0.054	2.5110	1200	0.3088
0.0527	2.7204	1300	0.3102
0.0529	2.9298	1400	0.3101

Framework versions

• PEFT 0.14.0
• Transformers 4.49.0
• Pytorch 2.6.0+cu126
• Datasets 3.3.2
• Tokenizers 0.21.0