lc_full_balanced

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5766

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss
1.6135	1.0	282	1.5535
1.7292	2.0	564	1.5380
1.502	3.0	846	1.5312
1.5672	4.0	1128	1.5347
1.3714	5.0	1410	1.5365
1.3268	6.0	1692	1.5415
1.6069	7.0	1974	1.5425
1.4057	8.0	2256	1.5483
1.3989	9.0	2538	1.5578
1.6766	10.0	2820	1.5601
1.4027	11.0	3102	1.5673
1.3554	12.0	3384	1.5703
1.3135	13.0	3666	1.5729
1.2954	14.0	3948	1.5745
1.2543	15.0	4230	1.5758
1.4603	16.0	4512	1.5763
1.2734	17.0	4794	1.5765
1.3247	18.0	5076	1.5764
1.2645	19.0	5358	1.5769
1.376	20.0	5640	1.5766

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.1.0+cu118
• Datasets 2.19.2
• Tokenizers 0.19.1