Hugging Face's logo

vichyt
/
codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_7

Generated from Trainer
Model card Files Community
codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_7 / README.md
vichyt's picture
vichyt
update model card README.md
5ff87bf
preview code
|
raw
history blame
No virus
11.7 kB
metadata
license: bsd-3-clause
base_model: Salesforce/codet5p-770m-py
tags:
  - generated_from_trainer
datasets:
  - mbpp
model-index:
  - name: codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_7
    results: []

codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_7

This model is a fine-tuned version of Salesforce/codet5p-770m-py on the mbpp dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7264
  • Codebleu: 0.1085
  • Ngram Match Score: 0.0170
  • Weighted Ngram Match Score: 0.0403
  • Syntax Match Score: 0.1283
  • Dataflow Match Score: 0.1285

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 64

Training results

Training Loss Epoch Step Validation Loss Codebleu Ngram Match Score Weighted Ngram Match Score Syntax Match Score Dataflow Match Score
0.9826 1.0 15 0.9245 0.0072 0.0000 0.0000 0.0079 0.0100
0.9647 2.0 30 0.9231 0.0072 0.0000 0.0000 0.0079 0.0100
0.9702 3.0 45 0.9200 0.0072 0.0000 0.0000 0.0079 0.0100
0.9452 4.0 60 0.9132 0.0080 0.0000 0.0000 0.0079 0.0120
0.9489 5.0 75 0.8993 0.0285 0.0000 0.0125 0.0238 0.0442
0.9429 6.0 90 0.8758 0.0629 0.0034 0.0295 0.0728 0.0763
0.927 7.0 105 0.8542 0.0967 0.0231 0.0527 0.1243 0.0984
0.8731 8.0 120 0.8435 0.0949 0.0224 0.0517 0.1243 0.0944
0.8557 9.0 135 0.8340 0.0948 0.0212 0.0518 0.1243 0.0944
0.8533 10.0 150 0.8233 0.0926 0.0192 0.0500 0.1177 0.0964
0.8407 11.0 165 0.8117 0.0991 0.0123 0.0371 0.1270 0.1084
0.8339 12.0 180 0.7991 0.0992 0.0113 0.0360 0.1257 0.1104
0.814 13.0 195 0.7891 0.0998 0.0104 0.0353 0.1217 0.1165
0.7913 14.0 210 0.7797 0.0930 0.0096 0.0341 0.1190 0.1024
0.803 15.0 225 0.7729 0.0968 0.0107 0.0347 0.1283 0.1024
0.8047 16.0 240 0.7674 0.0945 0.0110 0.0347 0.1243 0.1004
0.7931 17.0 255 0.7632 0.1013 0.0131 0.0368 0.1323 0.1084
0.7705 18.0 270 0.7592 0.1006 0.0143 0.0397 0.1296 0.1084
0.7823 19.0 285 0.7549 0.1006 0.0143 0.0397 0.1296 0.1084
0.765 20.0 300 0.7513 0.1042 0.0122 0.0352 0.1323 0.1165
0.7549 21.0 315 0.7478 0.1054 0.0127 0.0358 0.1310 0.1205
0.7592 22.0 330 0.7451 0.1055 0.0129 0.0358 0.1310 0.1205
0.754 23.0 345 0.7437 0.1055 0.0129 0.0358 0.1310 0.1205
0.7494 24.0 360 0.7425 0.1121 0.0133 0.0358 0.1376 0.1305
0.7748 25.0 375 0.7409 0.1121 0.0130 0.0352 0.1376 0.1305
0.7439 26.0 390 0.7393 0.1114 0.0138 0.0358 0.1376 0.1285
0.7302 27.0 405 0.7377 0.1123 0.0133 0.0352 0.1402 0.1285
0.737 28.0 420 0.7369 0.1124 0.0135 0.0352 0.1402 0.1285
0.7246 29.0 435 0.7369 0.1103 0.0138 0.0358 0.1349 0.1285
0.7269 30.0 450 0.7361 0.1103 0.0137 0.0352 0.1349 0.1285
0.7233 31.0 465 0.7346 0.1103 0.0137 0.0352 0.1349 0.1285
0.7276 32.0 480 0.7331 0.1077 0.0143 0.0356 0.1323 0.1245
0.7161 33.0 495 0.7323 0.1077 0.0141 0.0356 0.1323 0.1245
0.7294 34.0 510 0.7313 0.1077 0.0141 0.0356 0.1323 0.1245
0.7071 35.0 525 0.7303 0.1039 0.0125 0.0340 0.1257 0.1225
0.7358 36.0 540 0.7299 0.1039 0.0125 0.0340 0.1257 0.1225
0.7174 37.0 555 0.7299 0.1050 0.0125 0.0340 0.1283 0.1225
0.723 38.0 570 0.7304 0.1066 0.0129 0.0340 0.1283 0.1265
0.7125 39.0 585 0.7300 0.1066 0.0129 0.0340 0.1283 0.1265
0.7215 40.0 600 0.7294 0.1066 0.0129 0.0340 0.1283 0.1265
0.7039 41.0 615 0.7291 0.1066 0.0129 0.0340 0.1283 0.1265
0.7034 42.0 630 0.7287 0.1050 0.0128 0.0340 0.1283 0.1225
0.7063 43.0 645 0.7292 0.1050 0.0131 0.0340 0.1283 0.1225
0.7128 44.0 660 0.7297 0.1059 0.0136 0.0340 0.1283 0.1245
0.7065 45.0 675 0.7297 0.1059 0.0136 0.0340 0.1283 0.1245
0.7037 46.0 690 0.7289 0.1059 0.0136 0.0340 0.1283 0.1245
0.7103 47.0 705 0.7277 0.1059 0.0136 0.0340 0.1283 0.1245
0.7055 48.0 720 0.7275 0.1059 0.0134 0.0340 0.1283 0.1245
0.7018 49.0 735 0.7276 0.1075 0.0140 0.0340 0.1283 0.1285
0.6889 50.0 750 0.7281 0.1074 0.0134 0.0336 0.1283 0.1285
0.7159 51.0 765 0.7279 0.1074 0.0132 0.0336 0.1283 0.1285
0.7091 52.0 780 0.7276 0.1074 0.0132 0.0336 0.1283 0.1285
0.7064 53.0 795 0.7270 0.1085 0.0169 0.0403 0.1283 0.1285
0.6773 54.0 810 0.7267 0.1085 0.0169 0.0403 0.1283 0.1285
0.7052 55.0 825 0.7262 0.1085 0.0169 0.0403 0.1283 0.1285
0.7047 56.0 840 0.7265 0.1085 0.0169 0.0403 0.1283 0.1285
0.7119 57.0 855 0.7263 0.1085 0.0169 0.0403 0.1283 0.1285
0.6952 58.0 870 0.7263 0.1085 0.0169 0.0403 0.1283 0.1285
0.709 59.0 885 0.7262 0.1085 0.0169 0.0403 0.1283 0.1285
0.6899 60.0 900 0.7262 0.1085 0.0169 0.0403 0.1283 0.1285
0.7147 61.0 915 0.7264 0.1085 0.0169 0.0403 0.1283 0.1285
0.699 62.0 930 0.7264 0.1085 0.0170 0.0403 0.1283 0.1285
0.6958 63.0 945 0.7264 0.1085 0.0170 0.0403 0.1283 0.1285
0.69 64.0 960 0.7264 0.1085 0.0170 0.0403 0.1283 0.1285

Framework versions

  • Transformers 4.31.0
  • Pytorch 2.0.1
  • Datasets 2.14.4
  • Tokenizers 0.13.3