codet5p-770m-py-sanitized-codebleu-1-True-1e-07-0.1-prefix-tuning

This model is a fine-tuned version of Salesforce/codet5p-770m-py on the mbpp dataset. It achieves the following results on the evaluation set:

• Loss: 7.8250
• Codebleu: 0.0215
• Ngram Match Score: 0.0004
• Weighted Ngram Match Score: 0.0003
• Syntax Match Score: 0.0013
• Dataflow Match Score: 0.0522

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Codebleu	Ngram Match Score	Weighted Ngram Match Score	Syntax Match Score	Dataflow Match Score
7.9988	1.0	8	7.8277	0.0231	0.0004	0.0004	0.0013	0.0562
8.0077	2.0	16	7.8276	0.0231	0.0004	0.0004	0.0013	0.0562
8.0173	3.0	24	7.8276	0.0231	0.0004	0.0004	0.0013	0.0562
7.996	4.0	32	7.8276	0.0231	0.0004	0.0004	0.0013	0.0562
8.0369	5.0	40	7.8276	0.0231	0.0004	0.0004	0.0013	0.0562
8.0406	6.0	48	7.8276	0.0231	0.0004	0.0004	0.0013	0.0562
8.0162	7.0	56	7.8275	0.0231	0.0004	0.0004	0.0013	0.0562
7.9996	8.0	64	7.8274	0.0231	0.0004	0.0004	0.0013	0.0562
7.9955	9.0	72	7.8274	0.0231	0.0004	0.0004	0.0013	0.0562
8.015	10.0	80	7.8273	0.0231	0.0004	0.0004	0.0013	0.0562
8.0264	11.0	88	7.8272	0.0231	0.0004	0.0004	0.0013	0.0562
8.0091	12.0	96	7.8270	0.0215	0.0004	0.0003	0.0013	0.0522
8.0184	13.0	104	7.8269	0.0215	0.0004	0.0003	0.0013	0.0522
8.0261	14.0	112	7.8268	0.0215	0.0004	0.0003	0.0013	0.0522
7.9791	15.0	120	7.8267	0.0215	0.0004	0.0003	0.0013	0.0522
8.0159	16.0	128	7.8265	0.0215	0.0004	0.0003	0.0013	0.0522
7.9996	17.0	136	7.8264	0.0215	0.0004	0.0003	0.0013	0.0522
8.0404	18.0	144	7.8263	0.0215	0.0004	0.0003	0.0013	0.0522
7.9689	19.0	152	7.8262	0.0215	0.0004	0.0003	0.0013	0.0522
8.0391	20.0	160	7.8261	0.0215	0.0004	0.0003	0.0013	0.0522
7.9905	21.0	168	7.8260	0.0215	0.0004	0.0003	0.0013	0.0522
8.0239	22.0	176	7.8259	0.0215	0.0004	0.0003	0.0013	0.0522
8.0125	23.0	184	7.8258	0.0215	0.0004	0.0003	0.0013	0.0522
7.9988	24.0	192	7.8257	0.0215	0.0004	0.0003	0.0013	0.0522
7.9675	25.0	200	7.8257	0.0215	0.0004	0.0003	0.0013	0.0522
8.0127	26.0	208	7.8256	0.0215	0.0004	0.0003	0.0013	0.0522
8.0141	27.0	216	7.8255	0.0215	0.0004	0.0003	0.0013	0.0522
8.0028	28.0	224	7.8255	0.0215	0.0004	0.0003	0.0013	0.0522
8.0304	29.0	232	7.8254	0.0215	0.0004	0.0003	0.0013	0.0522
8.0433	30.0	240	7.8254	0.0215	0.0004	0.0003	0.0013	0.0522
8.0184	31.0	248	7.8253	0.0215	0.0004	0.0003	0.0013	0.0522
8.0456	32.0	256	7.8253	0.0215	0.0004	0.0003	0.0013	0.0522
8.0378	33.0	264	7.8252	0.0215	0.0004	0.0003	0.0013	0.0522
8.0035	34.0	272	7.8252	0.0215	0.0004	0.0003	0.0013	0.0522
8.0212	35.0	280	7.8252	0.0215	0.0004	0.0003	0.0013	0.0522
8.0033	36.0	288	7.8251	0.0215	0.0004	0.0003	0.0013	0.0522
8.0311	37.0	296	7.8251	0.0215	0.0004	0.0003	0.0013	0.0522
8.0264	38.0	304	7.8251	0.0215	0.0004	0.0003	0.0013	0.0522
7.9892	39.0	312	7.8251	0.0215	0.0004	0.0003	0.0013	0.0522
8.0047	40.0	320	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0111	41.0	328	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0124	42.0	336	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0408	43.0	344	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
7.9969	44.0	352	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0283	45.0	360	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0449	46.0	368	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0017	47.0	376	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0328	48.0	384	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
7.9923	49.0	392	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522
8.0266	50.0	400	7.8250	0.0215	0.0004	0.0003	0.0013	0.0522

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1
• Datasets 2.14.4
• Tokenizers 0.13.3