Hugging Face's logo

vichyt
/
codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_1

Generated from Trainer
Model card Files Community
codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_1 / README.md
vichyt's picture
vichyt
update model card README.md
3067584
preview code
|
raw
history blame
No virus
11.7 kB
metadata
license: bsd-3-clause
base_model: Salesforce/codet5p-770m-py
tags:
  - generated_from_trainer
datasets:
  - mbpp
model-index:
  - name: codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_1
    results: []

codet5p-770m-py-sanitized-codebleu-1-True-5e-05-0.1-lora-layer_1

This model is a fine-tuned version of Salesforce/codet5p-770m-py on the mbpp dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7230
  • Codebleu: 0.0971
  • Ngram Match Score: 0.0163
  • Weighted Ngram Match Score: 0.0411
  • Syntax Match Score: 0.1058
  • Dataflow Match Score: 0.1225

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 64

Training results

Training Loss Epoch Step Validation Loss Codebleu Ngram Match Score Weighted Ngram Match Score Syntax Match Score Dataflow Match Score
0.9754 1.0 15 0.9244 0.0072 0.0000 0.0000 0.0079 0.0100
0.9686 2.0 30 0.9226 0.0072 0.0000 0.0000 0.0079 0.0100
0.9714 3.0 45 0.9188 0.0080 0.0000 0.0000 0.0079 0.0120
0.9367 4.0 60 0.9105 0.0080 0.0000 0.0000 0.0079 0.0120
0.9426 5.0 75 0.8940 0.0361 0.0006 0.0169 0.0357 0.0502
0.9326 6.0 90 0.8707 0.0800 0.0189 0.0462 0.0952 0.0884
0.9114 7.0 105 0.8584 0.0965 0.0168 0.0464 0.1151 0.1104
0.8773 8.0 120 0.8524 0.1004 0.0178 0.0470 0.1243 0.1104
0.8673 9.0 135 0.8486 0.0932 0.0216 0.0481 0.1111 0.1044
0.8738 10.0 150 0.8444 0.0924 0.0242 0.0484 0.1085 0.1044
0.8685 11.0 165 0.8404 0.0936 0.0232 0.0485 0.1138 0.1024
0.8798 12.0 180 0.8350 0.0929 0.0237 0.0486 0.1138 0.1004
0.8548 13.0 195 0.8299 0.0951 0.0238 0.0495 0.1190 0.1004
0.8387 14.0 210 0.8216 0.0953 0.0244 0.0508 0.1190 0.1004
0.8486 15.0 225 0.8124 0.0967 0.0237 0.0526 0.1243 0.0984
0.8587 16.0 240 0.8042 0.0952 0.0228 0.0517 0.1270 0.0924
0.8377 17.0 255 0.7949 0.0968 0.0220 0.0520 0.1230 0.1004
0.8159 18.0 270 0.7868 0.0951 0.0224 0.0538 0.1204 0.0984
0.8246 19.0 285 0.7795 0.0975 0.0222 0.0538 0.1243 0.1004
0.8126 20.0 300 0.7711 0.0982 0.0209 0.0536 0.1243 0.1024
0.8009 21.0 315 0.7632 0.1001 0.0213 0.0538 0.1230 0.1084
0.802 22.0 330 0.7608 0.0985 0.0182 0.0512 0.1164 0.1124
0.7926 23.0 345 0.7598 0.0989 0.0171 0.0478 0.1124 0.1185
0.7966 24.0 360 0.7571 0.1026 0.0167 0.0480 0.1177 0.1225
0.8129 25.0 375 0.7538 0.1021 0.0173 0.0481 0.1164 0.1225
0.7831 26.0 390 0.7509 0.1029 0.0152 0.0451 0.1217 0.1205
0.7765 27.0 405 0.7482 0.0992 0.0141 0.0437 0.1190 0.1145
0.7814 28.0 420 0.7449 0.1040 0.0142 0.0436 0.1270 0.1185
0.7663 29.0 435 0.7426 0.1040 0.0142 0.0436 0.1270 0.1185
0.7641 30.0 450 0.7402 0.1043 0.0135 0.0393 0.1230 0.1245
0.768 31.0 465 0.7388 0.1053 0.0144 0.0408 0.1230 0.1265
0.7629 32.0 480 0.7372 0.1043 0.0144 0.0408 0.1204 0.1265
0.7536 33.0 495 0.7356 0.1101 0.0161 0.0471 0.1270 0.1325
0.7709 34.0 510 0.7342 0.1094 0.0166 0.0475 0.1270 0.1305
0.7521 35.0 525 0.7330 0.1092 0.0170 0.0476 0.1243 0.1325
0.7741 36.0 540 0.7316 0.1074 0.0171 0.0476 0.1217 0.1305
0.7614 37.0 555 0.7313 0.1087 0.0171 0.0476 0.1230 0.1325
0.7663 38.0 570 0.7307 0.1079 0.0173 0.0476 0.1230 0.1305
0.7562 39.0 585 0.7296 0.1079 0.0173 0.0476 0.1230 0.1305
0.7706 40.0 600 0.7292 0.1079 0.0173 0.0476 0.1230 0.1305
0.7512 41.0 615 0.7291 0.1079 0.0173 0.0476 0.1230 0.1305
0.7469 42.0 630 0.7288 0.1085 0.0178 0.0481 0.1243 0.1305
0.7544 43.0 645 0.7285 0.1081 0.0177 0.0466 0.1217 0.1325
0.7639 44.0 660 0.7279 0.1092 0.0179 0.0466 0.1243 0.1325
0.7505 45.0 675 0.7273 0.1120 0.0212 0.0528 0.1270 0.1345
0.7491 46.0 690 0.7265 0.1120 0.0212 0.0528 0.1270 0.1345
0.7525 47.0 705 0.7264 0.1087 0.0209 0.0522 0.1230 0.1305
0.7535 48.0 720 0.7262 0.1114 0.0205 0.0523 0.1257 0.1345
0.7596 49.0 735 0.7256 0.1124 0.0207 0.0523 0.1283 0.1345
0.7368 50.0 750 0.7253 0.1087 0.0209 0.0522 0.1230 0.1305
0.7649 51.0 765 0.7251 0.1053 0.0197 0.0507 0.1190 0.1265
0.7516 52.0 780 0.7248 0.1053 0.0197 0.0507 0.1190 0.1265
0.7592 53.0 795 0.7246 0.1038 0.0197 0.0494 0.1138 0.1285
0.7258 54.0 810 0.7240 0.1099 0.0202 0.0508 0.1204 0.1365
0.7524 55.0 825 0.7239 0.1038 0.0197 0.0494 0.1138 0.1285
0.7534 56.0 840 0.7237 0.0970 0.0162 0.0411 0.1058 0.1225
0.7583 57.0 855 0.7233 0.0968 0.0164 0.0411 0.1032 0.1245
0.7468 58.0 870 0.7232 0.0970 0.0162 0.0411 0.1058 0.1225
0.7567 59.0 885 0.7232 0.0970 0.0162 0.0411 0.1058 0.1225
0.7433 60.0 900 0.7232 0.0970 0.0162 0.0411 0.1058 0.1225
0.7569 61.0 915 0.7231 0.0971 0.0163 0.0411 0.1058 0.1225
0.747 62.0 930 0.7230 0.0971 0.0163 0.0411 0.1058 0.1225
0.7482 63.0 945 0.7230 0.0971 0.0163 0.0411 0.1058 0.1225
0.7471 64.0 960 0.7230 0.0971 0.0163 0.0411 0.1058 0.1225

Framework versions

  • Transformers 4.31.0
  • Pytorch 2.0.1
  • Datasets 2.14.4
  • Tokenizers 0.13.3