Hugging Face's logo

AI4PD
/
lact

Text Generation
Transformers
PyTorch
gpt2
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
1
lact / README.md
nferruz's picture
nferruz
Upload 13 files
4a15b34
preview code
|
raw
history blame
26.8 kB
metadata
tags:
  - generated_from_trainer
model-index:
  - name: output
    results: []

output

This model is a fine-tuned version of /home/woody/b114cb/b114cb10/zymCTRL/gpt2-large/config.json on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3014

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 8e-05
  • train_batch_size: 1
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10.0

Training results

Training Loss Epoch Step Validation Loss
3.2882 0.02 10 2.9581
2.5059 0.04 20 2.3844
2.3368 0.06 30 2.3644
2.3476 0.08 40 2.3494
2.3185 0.1 50 2.3697
2.3468 0.12 60 2.3255
2.262 0.14 70 2.2512
2.1646 0.16 80 2.1945
2.1558 0.18 90 2.1885
2.1934 0.2 100 2.1483
2.0855 0.22 110 2.1152
2.0844 0.24 120 2.0839
2.0647 0.26 130 2.0615
1.9665 0.28 140 2.0330
1.9761 0.3 150 2.0068
1.9428 0.32 160 1.9914
1.9351 0.34 170 1.9369
1.9366 0.36 180 1.9139
1.9548 0.38 190 1.8789
1.9625 0.4 200 1.8486
1.8584 0.42 210 1.8198
1.8857 0.44 220 1.8118
1.7574 0.46 230 1.7603
1.8114 0.48 240 1.7370
1.7303 0.5 250 1.7205
1.7535 0.52 260 1.7124
1.7775 0.54 270 1.7013
1.685 0.56 280 1.6612
1.5898 0.58 290 1.6578
1.7875 0.6 300 1.6458
1.628 0.62 310 1.6253
1.6186 0.64 320 1.6195
1.6899 0.66 330 1.6102
1.5908 0.68 340 1.5907
1.6514 0.7 350 1.6104
1.6027 0.72 360 1.5766
1.6319 0.74 370 1.5623
1.6103 0.76 380 1.5764
1.4518 0.78 390 1.5449
1.498 0.8 400 1.5345
1.5266 0.82 410 1.5413
1.5622 0.84 420 1.5229
1.4863 0.86 430 1.5208
1.5492 0.88 440 1.4996
1.5515 0.9 450 1.4857
1.4799 0.92 460 1.4935
1.4514 0.94 470 1.4745
1.5462 0.96 480 1.4784
1.6032 0.98 490 1.4911
1.7418 1.0 500 1.4733
1.4983 1.02 510 1.4646
1.5383 1.04 520 1.4442
1.3454 1.06 530 1.4332
1.3128 1.08 540 1.4261
1.5472 1.1 550 1.4232
1.252 1.12 560 1.3924
1.3538 1.14 570 1.3975
1.5448 1.16 580 1.3915
1.4016 1.18 590 1.4025
1.3041 1.2 600 1.3837
1.3857 1.22 610 1.3890
1.2923 1.24 620 1.3452
1.28 1.26 630 1.3492
1.4052 1.28 640 1.3254
1.3992 1.3 650 1.3670
1.5044 1.32 660 1.3153
1.2274 1.34 670 1.3142
1.2392 1.36 680 1.3150
1.365 1.38 690 1.2966
1.3024 1.4 700 1.2688
1.347 1.42 710 1.2874
1.3898 1.44 720 1.2543
1.4256 1.46 730 1.2397
1.2566 1.48 740 1.2430
1.2473 1.5 750 1.2135
1.1466 1.52 760 1.2171
1.3065 1.54 770 1.1897
1.3033 1.56 780 1.1646
1.1166 1.58 790 1.1723
1.0874 1.6 800 1.1511
1.017 1.62 810 1.1396
1.0437 1.64 820 1.1016
1.2206 1.66 830 1.0841
0.9738 1.68 840 1.0760
1.1351 1.7 850 1.0562
1.0697 1.72 860 1.0556
1.0296 1.74 870 1.0342
1.0904 1.76 880 1.0047
1.01 1.78 890 1.0184
0.951 1.8 900 0.9845
1.0111 1.82 910 0.9675
1.0824 1.84 920 0.9759
0.9745 1.86 930 0.9336
0.8632 1.88 940 0.9347
0.9959 1.9 950 0.9395
0.8906 1.92 960 0.8965
1.0552 1.94 970 0.8892
0.8387 1.96 980 0.8822
1.0068 1.98 990 0.8805
1.083 2.0 1000 0.8490
0.8407 2.02 1010 0.8457
0.7468 2.04 1020 0.8285
0.8421 2.06 1030 0.8055
0.8407 2.08 1040 0.8160
0.8126 2.1 1050 0.8266
0.7318 2.12 1060 0.8151
0.9142 2.14 1070 0.7876
0.6483 2.16 1080 0.7866
0.8092 2.18 1090 0.7818
0.8235 2.2 1100 0.7708
0.7062 2.22 1110 0.7693
0.7348 2.24 1120 0.7875
0.7507 2.26 1130 0.7567
0.7588 2.28 1140 0.7565
0.605 2.3 1150 0.7298
0.8721 2.32 1160 0.7254
0.6988 2.34 1170 0.7072
0.6294 2.36 1180 0.7082
0.7117 2.38 1190 0.7113
0.8558 2.4 1200 0.6991
0.6187 2.42 1210 0.6905
0.6791 2.44 1220 0.6875
0.5447 2.46 1230 0.6869
0.7299 2.48 1240 0.6777
0.5829 2.5 1250 0.6658
0.6435 2.52 1260 0.6603
0.7303 2.54 1270 0.6578
0.7244 2.56 1280 0.6594
0.6463 2.58 1290 0.6409
0.7766 2.6 1300 0.6417
0.6012 2.62 1310 0.6461
0.5974 2.64 1320 0.6365
0.556 2.66 1330 0.6301
0.6369 2.68 1340 0.6247
0.5699 2.7 1350 0.6163
0.624 2.72 1360 0.6138
0.6774 2.74 1370 0.6135
0.5553 2.76 1380 0.6076
0.604 2.78 1390 0.5938
0.6087 2.8 1400 0.5956
0.5935 2.82 1410 0.5933
0.6042 2.84 1420 0.5911
0.6425 2.86 1430 0.5844
0.6316 2.88 1440 0.5745
0.597 2.9 1450 0.5695
0.5754 2.92 1460 0.5704
0.5197 2.94 1470 0.5697
0.6256 2.96 1480 0.5596
0.5818 2.98 1490 0.5599
0.5464 3.01 1500 0.5565
0.4616 3.03 1510 0.5629
0.6482 3.05 1520 0.5529
0.5356 3.07 1530 0.5526
0.5688 3.09 1540 0.5528
0.6018 3.11 1550 0.5408
0.5794 3.13 1560 0.5371
0.5443 3.15 1570 0.5375
0.4435 3.17 1580 0.5345
0.5087 3.19 1590 0.5293
0.518 3.21 1600 0.5336
0.5914 3.23 1610 0.5316
0.5667 3.25 1620 0.5254
0.5218 3.27 1630 0.5207
0.4267 3.29 1640 0.5270
0.5839 3.31 1650 0.5199
0.5095 3.33 1660 0.5268
0.4616 3.35 1670 0.5192
0.5027 3.37 1680 0.5106
0.441 3.39 1690 0.5150
0.4416 3.41 1700 0.5156
0.4411 3.43 1710 0.5103
0.47 3.45 1720 0.5038
0.5079 3.47 1730 0.5048
0.3913 3.49 1740 0.5082
0.4977 3.51 1750 0.4976
0.5905 3.53 1760 0.4975
0.4362 3.55 1770 0.4962
0.4309 3.57 1780 0.5008
0.4477 3.59 1790 0.4988
0.4826 3.61 1800 0.4886
0.6181 3.63 1810 0.4885
0.4738 3.65 1820 0.4879
0.4932 3.67 1830 0.4818
0.4684 3.69 1840 0.4812
0.5484 3.71 1850 0.4767
0.5086 3.73 1860 0.4791
0.3548 3.75 1870 0.4793
0.5229 3.77 1880 0.4765
0.4578 3.79 1890 0.4704
0.5277 3.81 1900 0.4691
0.4683 3.83 1910 0.4649
0.448 3.85 1920 0.4684
0.3752 3.87 1930 0.4697
0.4631 3.89 1940 0.4678
0.4277 3.91 1950 0.4608
0.3646 3.93 1960 0.4609
0.5276 3.95 1970 0.4543
0.431 3.97 1980 0.4539
0.5465 3.99 1990 0.4550
0.4954 4.01 2000 0.4523
0.4886 4.03 2010 0.4499
0.4898 4.05 2020 0.4462
0.4072 4.07 2030 0.4479
0.4565 4.09 2040 0.4458
0.3739 4.11 2050 0.4475
0.4211 4.13 2060 0.4486
0.4048 4.15 2070 0.4393
0.5064 4.17 2080 0.4351
0.4652 4.19 2090 0.4379
0.4061 4.21 2100 0.4341
0.3784 4.23 2110 0.4390
0.4142 4.25 2120 0.4354
0.3625 4.27 2130 0.4415
0.3807 4.29 2140 0.4403
0.4154 4.31 2150 0.4308
0.4509 4.33 2160 0.4298
0.4254 4.35 2170 0.4239
0.4323 4.37 2180 0.4214
0.4359 4.39 2190 0.4291
0.3759 4.41 2200 0.4224
0.4534 4.43 2210 0.4225
0.4013 4.45 2220 0.4262
0.4331 4.47 2230 0.4214
0.4373 4.49 2240 0.4198
0.4975 4.51 2250 0.4236
0.423 4.53 2260 0.4189
0.4503 4.55 2270 0.4171
0.3796 4.57 2280 0.4172
0.4063 4.59 2290 0.4125
0.3841 4.61 2300 0.4119
0.2956 4.63 2310 0.4147
0.3486 4.65 2320 0.4246
0.3585 4.67 2330 0.4117
0.4496 4.69 2340 0.4091
0.399 4.71 2350 0.4049
0.3885 4.73 2360 0.4004
0.3728 4.75 2370 0.4003
0.2698 4.77 2380 0.4009
0.3799 4.79 2390 0.4003
0.4888 4.81 2400 0.3974
0.3795 4.83 2410 0.3995
0.4249 4.85 2420 0.3968
0.4635 4.87 2430 0.4001
0.4965 4.89 2440 0.3934
0.3745 4.91 2450 0.3987
0.3601 4.93 2460 0.3986
0.2878 4.95 2470 0.3941
0.4297 4.97 2480 0.3890
0.278 4.99 2490 0.3975
0.4509 5.01 2500 0.3907
0.3202 5.03 2510 0.3872
0.3047 5.05 2520 0.3956
0.2931 5.07 2530 0.3925
0.3487 5.09 2540 0.3910
0.2792 5.11 2550 0.3901
0.3446 5.13 2560 0.3873
0.3482 5.15 2570 0.3840
0.3464 5.17 2580 0.3835
0.3212 5.19 2590 0.3846
0.3847 5.21 2600 0.3819
0.3212 5.23 2610 0.3897
0.358 5.25 2620 0.3811
0.3471 5.27 2630 0.3805
0.3348 5.29 2640 0.3868
0.342 5.31 2650 0.3769
0.4504 5.33 2660 0.3774
0.2713 5.35 2670 0.3803
0.3848 5.37 2680 0.3776
0.354 5.39 2690 0.3758
0.3796 5.41 2700 0.3760
0.3654 5.43 2710 0.3737
0.3448 5.45 2720 0.3812
0.355 5.47 2730 0.3759
0.288 5.49 2740 0.3711
0.2991 5.51 2750 0.3691
0.3443 5.53 2760 0.3708
0.3374 5.55 2770 0.3659
0.4078 5.57 2780 0.3709
0.2967 5.59 2790 0.3683
0.3532 5.61 2800 0.3638
0.4123 5.63 2810 0.3642
0.3195 5.65 2820 0.3655
0.3161 5.67 2830 0.3599
0.4152 5.69 2840 0.3621
0.2802 5.71 2850 0.3648
0.2909 5.73 2860 0.3604
0.3105 5.75 2870 0.3604
0.3291 5.77 2880 0.3553
0.3916 5.79 2890 0.3603
0.3657 5.81 2900 0.3544
0.3745 5.83 2910 0.3559
0.3281 5.85 2920 0.3517
0.2892 5.87 2930 0.3551
0.4121 5.89 2940 0.3489
0.2908 5.91 2950 0.3532
0.3677 5.93 2960 0.3469
0.341 5.95 2970 0.3503
0.2319 5.97 2980 0.3497
0.2624 5.99 2990 0.3468
0.3324 6.01 3000 0.3480
0.2114 6.03 3010 0.3530
0.256 6.05 3020 0.3501
0.2716 6.07 3030 0.3490
0.2921 6.09 3040 0.3466
0.2924 6.11 3050 0.3531
0.3267 6.13 3060 0.3455
0.3488 6.15 3070 0.3428
0.301 6.17 3080 0.3455
0.2656 6.19 3090 0.3450
0.2377 6.21 3100 0.3474
0.2344 6.23 3110 0.3461
0.2816 6.25 3120 0.3489
0.2675 6.27 3130 0.3427
0.3315 6.29 3140 0.3393
0.335 6.31 3150 0.3406
0.2418 6.33 3160 0.3385
0.215 6.35 3170 0.3393
0.2279 6.37 3180 0.3427
0.2907 6.39 3190 0.3379
0.2184 6.41 3200 0.3438
0.3484 6.43 3210 0.3364
0.2327 6.45 3220 0.3406
0.2571 6.47 3230 0.3400
0.2864 6.49 3240 0.3367
0.2383 6.51 3250 0.3377
0.187 6.53 3260 0.3346
0.2453 6.55 3270 0.3349
0.296 6.57 3280 0.3339
0.2601 6.59 3290 0.3335
0.2927 6.61 3300 0.3340
0.2796 6.63 3310 0.3303
0.2393 6.65 3320 0.3351
0.2764 6.67 3330 0.3288
0.2547 6.69 3340 0.3327
0.3247 6.71 3350 0.3279
0.3217 6.73 3360 0.3283
0.2881 6.75 3370 0.3307
0.2897 6.77 3380 0.3281
0.3096 6.79 3390 0.3257
0.2463 6.81 3400 0.3244
0.2404 6.83 3410 0.3254
0.2907 6.85 3420 0.3227
0.2749 6.87 3430 0.3226
0.2262 6.89 3440 0.3226
0.2799 6.91 3450 0.3233
0.2764 6.93 3460 0.3198
0.2644 6.95 3470 0.3231
0.2733 6.97 3480 0.3188
0.2861 6.99 3490 0.3192
0.1757 7.01 3500 0.3243
0.2588 7.03 3510 0.3238
0.2132 7.05 3520 0.3207
0.2787 7.07 3530 0.3272
0.2786 7.09 3540 0.3229
0.2854 7.11 3550 0.3232
0.1982 7.13 3560 0.3237
0.2022 7.15 3570 0.3254
0.2592 7.17 3580 0.3258
0.2299 7.19 3590 0.3207
0.2054 7.21 3600 0.3197
0.208 7.23 3610 0.3216
0.2432 7.25 3620 0.3228
0.2452 7.27 3630 0.3181
0.264 7.29 3640 0.3238
0.2019 7.31 3650 0.3178
0.2299 7.33 3660 0.3218
0.2465 7.35 3670 0.3172
0.2466 7.37 3680 0.3167
0.2824 7.39 3690 0.3143
0.2314 7.41 3700 0.3143
0.2822 7.43 3710 0.3143
0.2254 7.45 3720 0.3139
0.2454 7.47 3730 0.3218
0.2656 7.49 3740 0.3116
0.2172 7.51 3750 0.3154
0.2408 7.53 3760 0.3127
0.1761 7.55 3770 0.3149
0.2232 7.57 3780 0.3114
0.2902 7.59 3790 0.3136
0.2485 7.61 3800 0.3146
0.1901 7.63 3810 0.3094
0.2962 7.65 3820 0.3120
0.2093 7.67 3830 0.3133
0.368 7.69 3840 0.3064
0.2849 7.71 3850 0.3091
0.1948 7.73 3860 0.3075
0.2241 7.75 3870 0.3078
0.1935 7.77 3880 0.3045
0.2045 7.79 3890 0.3065
0.159 7.81 3900 0.3082
0.1714 7.83 3910 0.3057
0.1984 7.85 3920 0.3059
0.2397 7.87 3930 0.3037
0.1884 7.89 3940 0.3054
0.2585 7.91 3950 0.3030
0.2476 7.93 3960 0.3058
0.2525 7.95 3970 0.3033
0.2001 7.97 3980 0.3062
0.1985 7.99 3990 0.3039
0.1984 8.02 4000 0.3139
0.2008 8.04 4010 0.3099
0.2159 8.06 4020 0.3085
0.2305 8.08 4030 0.3108
0.2007 8.1 4040 0.3050
0.2124 8.12 4050 0.3115
0.1435 8.14 4060 0.3084
0.1968 8.16 4070 0.3087
0.2507 8.18 4080 0.3084
0.1703 8.2 4090 0.3061
0.2511 8.22 4100 0.3106
0.1698 8.24 4110 0.3134
0.2518 8.26 4120 0.3101
0.1489 8.28 4130 0.3090
0.1759 8.3 4140 0.3098
0.1939 8.32 4150 0.3056
0.2168 8.34 4160 0.3106
0.2119 8.36 4170 0.3051
0.1793 8.38 4180 0.3056
0.2434 8.4 4190 0.3050
0.2601 8.42 4200 0.3065
0.1791 8.44 4210 0.3051
0.1404 8.46 4220 0.3058
0.222 8.48 4230 0.3059
0.1809 8.5 4240 0.3070
0.1745 8.52 4250 0.3066
0.2236 8.54 4260 0.3012
0.1965 8.56 4270 0.3037
0.1836 8.58 4280 0.3051
0.1912 8.6 4290 0.3017
0.2207 8.62 4300 0.3025
0.2481 8.64 4310 0.2997
0.1506 8.66 4320 0.3003
0.2216 8.68 4330 0.3035
0.1866 8.7 4340 0.3014
0.2025 8.72 4350 0.3035
0.1521 8.74 4360 0.2992
0.1598 8.76 4370 0.3034
0.185 8.78 4380 0.3017
0.2427 8.8 4390 0.2972
0.2343 8.82 4400 0.2979
0.1994 8.84 4410 0.2994
0.2671 8.86 4420 0.2986
0.1158 8.88 4430 0.2991
0.2127 8.9 4440 0.3000
0.1691 8.92 4450 0.2981
0.2103 8.94 4460 0.2979
0.1392 8.96 4470 0.2982
0.1712 8.98 4480 0.2943
0.2435 9.0 4490 0.2958
0.1715 9.02 4500 0.3055
0.1641 9.04 4510 0.3048
0.1529 9.06 4520 0.3029
0.1566 9.08 4530 0.3047
0.1382 9.1 4540 0.3027
0.1605 9.12 4550 0.3023
0.2167 9.14 4560 0.3055
0.1506 9.16 4570 0.3037
0.192 9.18 4580 0.3039
0.139 9.2 4590 0.3030
0.1974 9.22 4600 0.3038
0.167 9.24 4610 0.3037
0.2409 9.26 4620 0.3034
0.1494 9.28 4630 0.3048
0.1762 9.3 4640 0.3037
0.183 9.32 4650 0.3042
0.1773 9.34 4660 0.3043
0.1509 9.36 4670 0.3053
0.1994 9.38 4680 0.3045
0.1928 9.4 4690 0.3036
0.1158 9.42 4700 0.3038
0.1503 9.44 4710 0.3019
0.1556 9.46 4720 0.3029
0.1327 9.48 4730 0.3050
0.1772 9.5 4740 0.3057
0.1555 9.52 4750 0.3028
0.1363 9.54 4760 0.3014
0.139 9.56 4770 0.3010
0.1639 9.58 4780 0.3013
0.1669 9.6 4790 0.3015
0.144 9.62 4800 0.3023
0.1925 9.64 4810 0.3034
0.1615 9.66 4820 0.3025
0.1625 9.68 4830 0.3019
0.1355 9.7 4840 0.3023
0.1671 9.72 4850 0.3019
0.1447 9.74 4860 0.3021
0.1465 9.76 4870 0.3024
0.1794 9.78 4880 0.3021
0.156 9.8 4890 0.3011
0.1018 9.82 4900 0.3005
0.1403 9.84 4910 0.3011
0.1126 9.86 4920 0.3006
0.1595 9.88 4930 0.3007
0.1415 9.9 4940 0.3012
0.1651 9.92 4950 0.3015
0.1558 9.94 4960 0.3015
0.1734 9.96 4970 0.3014
0.1909 9.98 4980 0.3014
0.1246 10.0 4990 0.3014

Framework versions

  • Transformers 4.26.1
  • Pytorch 1.12.1+cu116
  • Datasets 2.10.0
  • Tokenizers 0.12.1