peft-dialogue-summary-training-1722323475

This model is a fine-tuned version of microsoft/phi-2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3175

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss
1.6386	0.0500	25	1.3931
1.1913	0.1001	50	1.3853
1.4455	0.1501	75	1.3551
1.2051	0.2001	100	1.3615
1.4377	0.2501	125	1.3421
1.1359	0.3002	150	1.3620
1.4014	0.3502	175	1.3393
1.1479	0.4002	200	1.3471
1.4439	0.4502	225	1.3337
1.2241	0.5003	250	1.3379
1.4578	0.5503	275	1.3327
1.1638	0.6003	300	1.3351
1.4283	0.6503	325	1.3301
1.2007	0.7004	350	1.3300
1.3962	0.7504	375	1.3276
1.1819	0.8004	400	1.3338
1.4409	0.8504	425	1.3272
1.196	0.9005	450	1.3281
1.4465	0.9505	475	1.3246
1.1907	1.0005	500	1.3272
1.4095	1.0505	525	1.3236
1.1451	1.1006	550	1.3245
1.346	1.1506	575	1.3230
1.1855	1.2006	600	1.3240
1.3688	1.2506	625	1.3220
1.1166	1.3007	650	1.3235
1.3762	1.3507	675	1.3210
1.1249	1.4007	700	1.3227
1.4183	1.4507	725	1.3205
1.1583	1.5008	750	1.3198
1.4363	1.5508	775	1.3190
1.1389	1.6008	800	1.3194
1.399	1.6508	825	1.3184
1.1295	1.7009	850	1.3195
1.451	1.7509	875	1.3179
1.143	1.8009	900	1.3177
1.3621	1.8509	925	1.3175
1.1494	1.9010	950	1.3176
1.3383	1.9510	975	1.3175
1.1089	2.0010	1000	1.3175

Framework versions

• PEFT 0.12.0
• Transformers 4.43.3
• Pytorch 2.4.0
• Datasets 2.20.0
• Tokenizers 0.19.1