with_cnn_summary_train_1

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4987

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.3255	0.9991	579	1.2416
1.2371	2.0	1159	1.2072
1.1535	2.9991	1738	1.2081
1.0356	4.0	2318	1.2215
0.9171	4.9991	2897	1.2679
0.7819	6.0	3477	1.3105
0.6835	6.9991	4056	1.3958
0.6058	8.0	4636	1.4633
0.4779	8.9991	5215	1.5415
0.3952	10.0	5795	1.6919
0.3454	10.9991	6374	1.8169
0.2743	12.0	6954	1.9270
0.2336	12.9991	7533	1.9968
0.2069	14.0	8113	2.1474
0.165	14.9991	8692	2.2376
0.1495	16.0	9272	2.3137
0.132	16.9991	9851	2.3877
0.125	18.0	10431	2.4865
0.1132	18.9991	11010	2.5050
0.114	19.9827	11580	2.4987

Framework versions

• PEFT 0.13.1
• Transformers 4.43.3
• Pytorch 2.4.0+cu121
• Datasets 3.0.1
• Tokenizers 0.19.1