README.md

metadata

tags:
  - generated_from_trainer
metrics:
  - accuracy
  - f1
model-index:
  - name: CommitPredictor
    results: []

CommitPredictor

This model is a fine-tuned version of microsoft/codebert-base-mlm on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4812
• Accuracy: 0.8993
• F1: 0.8993
• Bleu4: 0.9483

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Bleu4
1.1319	1.0	687	0.6982	0.8562	0.8562	0.8551
0.7784	2.0	1374	0.6501	0.8665	0.8665	0.8977
0.6779	3.0	2061	0.6229	0.8733	0.8733	0.8535
0.6579	4.0	2748	0.5978	0.8769	0.8769	0.9176
0.6319	5.0	3435	0.5833	0.8808	0.8808	0.8073
0.5988	6.0	4122	0.5627	0.8834	0.8834	0.9241
0.5939	7.0	4809	0.5533	0.8864	0.8864	0.9212
0.575	8.0	5496	0.5512	0.8860	0.8860	0.7943
0.5574	9.0	6183	0.5412	0.8879	0.8879	0.9396
0.553	10.0	6870	0.5276	0.8899	0.8899	0.8301
0.5371	11.0	7557	0.5341	0.8893	0.8893	0.9350
0.5302	12.0	8244	0.5236	0.8909	0.8909	0.8813
0.5245	13.0	8931	0.5153	0.8933	0.8933	0.8817
0.5165	14.0	9618	0.5138	0.8926	0.8926	0.9174
0.5122	15.0	10305	0.5144	0.8930	0.8930	0.8318
0.5007	16.0	10992	0.5007	0.8957	0.8957	0.9350
0.4954	17.0	11679	0.5041	0.8960	0.8960	0.9355
0.4894	18.0	12366	0.5000	0.8967	0.8967	0.7818
0.4851	19.0	13053	0.4915	0.8982	0.8982	0.9190
0.483	20.0	13740	0.4970	0.8962	0.8962	0.9359
0.4792	21.0	14427	0.4849	0.8971	0.8971	0.8458
0.4716	22.0	15114	0.4809	0.8990	0.8990	0.9367
0.4691	23.0	15801	0.4732	0.9006	0.9006	0.9478
0.4675	24.0	16488	0.4805	0.8989	0.8989	0.9412
0.4618	25.0	17175	0.4837	0.8997	0.8997	0.8373
0.4633	26.0	17862	0.4812	0.8993	0.8993	0.9483

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu117
• Datasets 2.7.1
• Tokenizers 0.13.2