README.md

metadata

tags:
  - generated_from_trainer
metrics:
  - accuracy
  - f1
model-index:
  - name: CommitPredictor
    results: []

CommitPredictor

This model is a fine-tuned version of microsoft/codebert-base-mlm on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4811
• Accuracy: 0.8991
• F1: 0.8991
• Bleu4: 0.9479

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Bleu4
1.143	1.0	687	0.6993	0.8563	0.8563	0.8531
0.7772	2.0	1374	0.6482	0.8677	0.8677	0.9036
0.6738	3.0	2061	0.6211	0.8734	0.8734	0.8189
0.6544	4.0	2748	0.5942	0.8782	0.8782	0.9196
0.6295	5.0	3435	0.5805	0.8815	0.8815	0.8079
0.5966	6.0	4122	0.5609	0.8838	0.8838	0.8186
0.5916	7.0	4809	0.5514	0.8870	0.8870	0.9103
0.5732	8.0	5496	0.5492	0.8861	0.8861	0.8067
0.5559	9.0	6183	0.5389	0.8881	0.8881	0.9353
0.5511	10.0	6870	0.5257	0.8901	0.8901	0.9297
0.5345	11.0	7557	0.5319	0.8905	0.8905	0.9363
0.5287	12.0	8244	0.5220	0.8911	0.8911	0.8816
0.5226	13.0	8931	0.5139	0.8938	0.8938	0.9438
0.5147	14.0	9618	0.5124	0.8929	0.8929	0.9145
0.511	15.0	10305	0.5131	0.8932	0.8932	0.8570
0.4996	16.0	10992	0.4997	0.8964	0.8964	0.9287
0.4949	17.0	11679	0.5033	0.8958	0.8958	0.9460
0.4882	18.0	12366	0.5003	0.8971	0.8971	0.7739
0.4837	19.0	13053	0.4914	0.8979	0.8979	0.9014
0.4822	20.0	13740	0.4962	0.8963	0.8963	0.9330
0.4778	21.0	14427	0.4844	0.8971	0.8971	0.8454
0.4704	22.0	15114	0.4809	0.8988	0.8988	0.9274
0.4676	23.0	15801	0.4735	0.9009	0.9009	0.9445
0.4663	24.0	16488	0.4792	0.8990	0.8990	0.9001
0.4605	25.0	17175	0.4826	0.8995	0.8995	0.8313
0.4621	26.0	17862	0.4811	0.8991	0.8991	0.9479

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu117
• Datasets 2.7.1
• Tokenizers 0.13.2