Hugging Face's logo

vichyt
/
codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_10

Generated from Trainer
Model card Files Community
codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_10 / README.md
vichyt's picture
vichyt
update model card README.md
01bdddb
preview code
|
raw
history blame
No virus
11.7 kB
metadata
license: bsd-3-clause
base_model: Salesforce/codet5p-770m-py
tags:
  - generated_from_trainer
datasets:
  - mbpp
model-index:
  - name: codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_10
    results: []

codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_10

This model is a fine-tuned version of Salesforce/codet5p-770m-py on the mbpp dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7219
  • Codebleu: 0.1061
  • Ngram Match Score: 0.0202
  • Weighted Ngram Match Score: 0.0460
  • Syntax Match Score: 0.1283
  • Dataflow Match Score: 0.1205

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 64

Training results

Training Loss Epoch Step Validation Loss Codebleu Ngram Match Score Weighted Ngram Match Score Syntax Match Score Dataflow Match Score
0.984 1.0 15 0.9245 0.0072 0.0000 0.0000 0.0079 0.0100
0.9647 2.0 30 0.9229 0.0072 0.0000 0.0000 0.0079 0.0100
0.9735 3.0 45 0.9191 0.0080 0.0000 0.0000 0.0079 0.0120
0.9397 4.0 60 0.9110 0.0080 0.0000 0.0000 0.0079 0.0120
0.9453 5.0 75 0.8945 0.0337 0.0002 0.0146 0.0344 0.0462
0.9376 6.0 90 0.8688 0.0807 0.0109 0.0349 0.0939 0.0964
0.9196 7.0 105 0.8494 0.1005 0.0200 0.0510 0.1270 0.1064
0.8634 8.0 120 0.8375 0.1001 0.0150 0.0392 0.1204 0.1165
0.8494 9.0 135 0.8262 0.0977 0.0135 0.0369 0.1151 0.1165
0.843 10.0 150 0.8134 0.0997 0.0135 0.0356 0.1164 0.1205
0.8304 11.0 165 0.8006 0.1071 0.0129 0.0361 0.1310 0.1245
0.8291 12.0 180 0.7911 0.1126 0.0158 0.0454 0.1376 0.1285
0.811 13.0 195 0.7835 0.1124 0.0147 0.0447 0.1376 0.1285
0.7917 14.0 210 0.7770 0.1122 0.0129 0.0444 0.1376 0.1285
0.7976 15.0 225 0.7716 0.1108 0.0154 0.0444 0.1296 0.1325
0.7978 16.0 240 0.7678 0.1049 0.0159 0.0427 0.1190 0.1285
0.7899 17.0 255 0.7642 0.1008 0.0140 0.0386 0.1164 0.1225
0.7663 18.0 270 0.7616 0.1051 0.0133 0.0369 0.1217 0.1285
0.7762 19.0 285 0.7572 0.1011 0.0110 0.0279 0.1124 0.1305
0.7656 20.0 300 0.7539 0.0938 0.0104 0.0279 0.1045 0.1205
0.7518 21.0 315 0.7509 0.0967 0.0100 0.0278 0.1098 0.1225
0.7484 22.0 330 0.7481 0.1046 0.0142 0.0363 0.1204 0.1285
0.7448 23.0 345 0.7464 0.0990 0.0141 0.0361 0.1124 0.1225
0.7439 24.0 360 0.7448 0.0964 0.0100 0.0277 0.1071 0.1245
0.77 25.0 375 0.7420 0.1022 0.0107 0.0295 0.1190 0.1265
0.7379 26.0 390 0.7410 0.0999 0.0146 0.0368 0.1124 0.1245
0.7246 27.0 405 0.7400 0.0953 0.0120 0.0303 0.1111 0.1165
0.738 28.0 420 0.7386 0.0969 0.0112 0.0292 0.1138 0.1185
0.7263 29.0 435 0.7378 0.0955 0.0125 0.0296 0.1138 0.1145
0.7185 30.0 450 0.7369 0.1007 0.0173 0.0391 0.1151 0.1225
0.7167 31.0 465 0.7362 0.1007 0.0177 0.0391 0.1151 0.1225
0.7191 32.0 480 0.7346 0.1044 0.0181 0.0410 0.1257 0.1205
0.7128 33.0 495 0.7335 0.1051 0.0179 0.0407 0.1257 0.1225
0.7156 34.0 510 0.7324 0.0980 0.0132 0.0322 0.1151 0.1185
0.698 35.0 525 0.7314 0.0980 0.0132 0.0322 0.1151 0.1185
0.7216 36.0 540 0.7312 0.1022 0.0137 0.0339 0.1230 0.1205
0.7113 37.0 555 0.7308 0.1014 0.0145 0.0337 0.1230 0.1185
0.7079 38.0 570 0.7305 0.1017 0.0200 0.0422 0.1243 0.1145
0.7065 39.0 585 0.7294 0.1024 0.0179 0.0403 0.1230 0.1185
0.7148 40.0 600 0.7286 0.1032 0.0174 0.0405 0.1230 0.1205
0.695 41.0 615 0.7281 0.1032 0.0175 0.0405 0.1230 0.1205
0.6948 42.0 630 0.7275 0.1025 0.0179 0.0408 0.1230 0.1185
0.6973 43.0 645 0.7269 0.1045 0.0183 0.0421 0.1257 0.1205
0.704 44.0 660 0.7267 0.1037 0.0187 0.0419 0.1257 0.1185
0.7002 45.0 675 0.7262 0.1005 0.0183 0.0422 0.1217 0.1145
0.6908 46.0 690 0.7258 0.1055 0.0192 0.0429 0.1296 0.1185
0.7007 47.0 705 0.7256 0.1044 0.0192 0.0429 0.1270 0.1185
0.693 48.0 720 0.7255 0.1044 0.0192 0.0429 0.1270 0.1185
0.6955 49.0 735 0.7250 0.1052 0.0191 0.0426 0.1270 0.1205
0.6849 50.0 750 0.7249 0.1058 0.0196 0.0461 0.1257 0.1225
0.7053 51.0 765 0.7246 0.1058 0.0196 0.0461 0.1257 0.1225
0.7013 52.0 780 0.7241 0.1053 0.0178 0.0425 0.1257 0.1225
0.7008 53.0 795 0.7236 0.1053 0.0180 0.0425 0.1257 0.1225
0.6707 54.0 810 0.7233 0.1053 0.0180 0.0425 0.1257 0.1225
0.6929 55.0 825 0.7228 0.1064 0.0180 0.0425 0.1283 0.1225
0.699 56.0 840 0.7228 0.1071 0.0176 0.0422 0.1283 0.1245
0.7078 57.0 855 0.7224 0.1064 0.0180 0.0425 0.1283 0.1225
0.6849 58.0 870 0.7222 0.1061 0.0202 0.0460 0.1283 0.1205
0.7021 59.0 885 0.7220 0.1061 0.0202 0.0460 0.1283 0.1205
0.6795 60.0 900 0.7220 0.1061 0.0202 0.0460 0.1283 0.1205
0.703 61.0 915 0.7220 0.1061 0.0202 0.0460 0.1283 0.1205
0.6849 62.0 930 0.7220 0.1061 0.0202 0.0460 0.1283 0.1205
0.6889 63.0 945 0.7219 0.1061 0.0202 0.0460 0.1283 0.1205
0.6813 64.0 960 0.7219 0.1061 0.0202 0.0460 0.1283 0.1205

Framework versions

  • Transformers 4.31.0
  • Pytorch 2.0.1
  • Datasets 2.14.4
  • Tokenizers 0.13.3