Edit model card

codet5Base-Doc-step-1

This model is a fine-tuned version of Salesforce/codet5-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0759

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss
No log 1.0 230 1.5475
No log 2.0 460 1.2669
2.0737 3.0 690 1.1299
2.0737 4.0 920 1.0710
1.0726 5.0 1150 1.0345
1.0726 6.0 1380 1.0217
0.7523 7.0 1610 1.0132
0.7523 8.0 1840 1.0215
0.544 9.0 2070 0.9870
0.544 10.0 2300 0.9926
0.4227 11.0 2530 1.0094
0.4227 12.0 2760 1.0106
0.4227 13.0 2990 1.0087
0.3384 14.0 3220 1.0196
0.3384 15.0 3450 1.0264
0.2754 16.0 3680 1.0283
0.2754 17.0 3910 1.0341
0.2371 18.0 4140 1.0493
0.2371 19.0 4370 1.0506
0.207 20.0 4600 1.0526
0.207 21.0 4830 1.0518
0.1761 22.0 5060 1.0600
0.1761 23.0 5290 1.0644
0.1735 24.0 5520 1.0668
0.1735 25.0 5750 1.0635
0.1735 26.0 5980 1.0726
0.1487 27.0 6210 1.0711
0.1487 28.0 6440 1.0760
0.1451 29.0 6670 1.0757
0.1451 30.0 6900 1.0759

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.1
Downloads last month
2
Safetensors
Model size
223M params
Tensor type
F32
·

Finetuned from