--- license: apache-2.0 tags: - generated_from_trainer model-index: - name: t5-end2end-questions-generation_3 results: [] --- # t5-end2end-questions-generation_3 This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.3421 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 1 - eval_batch_size: 1 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 4 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 7 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:----:|:---------------:| | 2.4349 | 0.02 | 10 | 1.7161 | | 1.7218 | 0.04 | 20 | 1.2939 | | 1.5305 | 0.06 | 30 | 1.0380 | | 1.1108 | 0.08 | 40 | 0.8948 | | 1.0712 | 0.09 | 50 | 0.7878 | | 0.939 | 0.11 | 60 | 0.7081 | | 0.8184 | 0.13 | 70 | 0.6778 | | 0.7966 | 0.15 | 80 | 0.6776 | | 0.765 | 0.17 | 90 | 0.6301 | | 0.6124 | 0.19 | 100 | 0.5941 | | 0.6417 | 0.21 | 110 | 0.5919 | | 0.6934 | 0.23 | 120 | 0.5772 | | 0.6427 | 0.24 | 130 | 0.5463 | | 0.6064 | 0.26 | 140 | 0.5295 | | 0.6152 | 0.28 | 150 | 0.5302 | | 0.5896 | 0.3 | 160 | 0.5361 | | 0.6301 | 0.32 | 170 | 0.5405 | | 0.563 | 0.34 | 180 | 0.5193 | | 0.5129 | 0.36 | 190 | 0.5242 | | 0.5303 | 0.38 | 200 | 0.5198 | | 0.5461 | 0.39 | 210 | 0.5311 | | 0.5553 | 0.41 | 220 | 0.5219 | | 0.5699 | 0.43 | 230 | 0.5079 | | 0.5846 | 0.45 | 240 | 0.4744 | | 0.4582 | 0.47 | 250 | 0.4608 | | 0.4997 | 0.49 | 260 | 0.4582 | | 0.4826 | 0.51 | 270 | 0.4629 | | 0.5534 | 0.53 | 280 | 0.4845 | | 0.4457 | 0.54 | 290 | 0.4842 | | 0.5113 | 0.56 | 300 | 0.4844 | | 0.5168 | 0.58 | 310 | 0.4942 | | 0.4885 | 0.6 | 320 | 0.5091 | | 0.5165 | 0.62 | 330 | 0.4980 | | 0.5986 | 0.64 | 340 | 0.4869 | | 0.5048 | 0.66 | 350 | 0.4775 | | 0.4521 | 0.68 | 360 | 0.4740 | | 0.5259 | 0.69 | 370 | 0.4634 | | 0.4904 | 0.71 | 380 | 0.4575 | | 0.4734 | 0.73 | 390 | 0.4448 | | 0.449 | 0.75 | 400 | 0.4375 | | 0.435 | 0.77 | 410 | 0.4237 | | 0.4791 | 0.79 | 420 | 0.4295 | | 0.4588 | 0.81 | 430 | 0.4318 | | 0.5415 | 0.83 | 440 | 0.4115 | | 0.4639 | 0.85 | 450 | 0.4060 | | 0.4976 | 0.86 | 460 | 0.4092 | | 0.4412 | 0.88 | 470 | 0.4115 | | 0.4465 | 0.9 | 480 | 0.4272 | | 0.53 | 0.92 | 490 | 0.4298 | | 0.4359 | 0.94 | 500 | 0.4297 | | 0.4071 | 0.96 | 510 | 0.4356 | | 0.463 | 0.98 | 520 | 0.4465 | | 0.5354 | 1.0 | 530 | 0.4133 | | 0.448 | 1.01 | 540 | 0.4079 | | 0.4019 | 1.03 | 550 | 0.4122 | | 0.4335 | 1.05 | 560 | 0.4000 | | 0.3879 | 1.07 | 570 | 0.3944 | | 0.4275 | 1.09 | 580 | 0.4044 | | 0.4506 | 1.11 | 590 | 0.4125 | | 0.3932 | 1.13 | 600 | 0.4000 | | 0.4415 | 1.15 | 610 | 0.4032 | | 0.3917 | 1.16 | 620 | 0.4087 | | 0.4292 | 1.18 | 630 | 0.4105 | | 0.459 | 1.2 | 640 | 0.4092 | | 0.3774 | 1.22 | 650 | 0.4103 | | 0.3645 | 1.24 | 660 | 0.4071 | | 0.3961 | 1.26 | 670 | 0.4032 | | 0.4685 | 1.28 | 680 | 0.4161 | | 0.4267 | 1.3 | 690 | 0.4157 | | 0.416 | 1.31 | 700 | 0.4039 | | 0.4514 | 1.33 | 710 | 0.4065 | | 0.3793 | 1.35 | 720 | 0.4084 | | 0.4009 | 1.37 | 730 | 0.4018 | | 0.4411 | 1.39 | 740 | 0.4060 | | 0.4498 | 1.41 | 750 | 0.4203 | | 0.3846 | 1.43 | 760 | 0.4247 | | 0.4194 | 1.45 | 770 | 0.4192 | | 0.3542 | 1.46 | 780 | 0.4225 | | 0.4442 | 1.48 | 790 | 0.4174 | | 0.4338 | 1.5 | 800 | 0.4042 | | 0.3626 | 1.52 | 810 | 0.4054 | | 0.4171 | 1.54 | 820 | 0.4104 | | 0.366 | 1.56 | 830 | 0.4074 | | 0.3463 | 1.58 | 840 | 0.3939 | | 0.4783 | 1.6 | 850 | 0.3854 | | 0.4476 | 1.62 | 860 | 0.3859 | | 0.4233 | 1.63 | 870 | 0.4041 | | 0.4241 | 1.65 | 880 | 0.4253 | | 0.3957 | 1.67 | 890 | 0.4263 | | 0.4751 | 1.69 | 900 | 0.4105 | | 0.4266 | 1.71 | 910 | 0.4023 | | 0.3685 | 1.73 | 920 | 0.4026 | | 0.4025 | 1.75 | 930 | 0.3939 | | 0.4099 | 1.77 | 940 | 0.3872 | | 0.4008 | 1.78 | 950 | 0.3813 | | 0.3646 | 1.8 | 960 | 0.3872 | | 0.4462 | 1.82 | 970 | 0.3894 | | 0.3752 | 1.84 | 980 | 0.3824 | | 0.478 | 1.86 | 990 | 0.3782 | | 0.4349 | 1.88 | 1000 | 0.3748 | | 0.3901 | 1.9 | 1010 | 0.3754 | | 0.4174 | 1.92 | 1020 | 0.3828 | | 0.4288 | 1.93 | 1030 | 0.3859 | | 0.4043 | 1.95 | 1040 | 0.3789 | | 0.3998 | 1.97 | 1050 | 0.3839 | | 0.424 | 1.99 | 1060 | 0.3940 | | 0.383 | 2.01 | 1070 | 0.3959 | | 0.3123 | 2.03 | 1080 | 0.3920 | | 0.3998 | 2.05 | 1090 | 0.3883 | | 0.4041 | 2.07 | 1100 | 0.3905 | | 0.3889 | 2.08 | 1110 | 0.3844 | | 0.4058 | 2.1 | 1120 | 0.3797 | | 0.3543 | 2.12 | 1130 | 0.3837 | | 0.3717 | 2.14 | 1140 | 0.3904 | | 0.3966 | 2.16 | 1150 | 0.3824 | | 0.3744 | 2.18 | 1160 | 0.3759 | | 0.331 | 2.2 | 1170 | 0.3772 | | 0.3582 | 2.22 | 1180 | 0.3756 | | 0.3639 | 2.23 | 1190 | 0.3782 | | 0.4402 | 2.25 | 1200 | 0.3825 | | 0.3617 | 2.27 | 1210 | 0.3843 | | 0.3748 | 2.29 | 1220 | 0.3897 | | 0.3651 | 2.31 | 1230 | 0.3875 | | 0.3694 | 2.33 | 1240 | 0.3842 | | 0.4077 | 2.35 | 1250 | 0.3844 | | 0.4 | 2.37 | 1260 | 0.3929 | | 0.3299 | 2.38 | 1270 | 0.3984 | | 0.3511 | 2.4 | 1280 | 0.3991 | | 0.4368 | 2.42 | 1290 | 0.3872 | | 0.4289 | 2.44 | 1300 | 0.3822 | | 0.3744 | 2.46 | 1310 | 0.3815 | | 0.3531 | 2.48 | 1320 | 0.3864 | | 0.3567 | 2.5 | 1330 | 0.3910 | | 0.4227 | 2.52 | 1340 | 0.3956 | | 0.3903 | 2.54 | 1350 | 0.3963 | | 0.3924 | 2.55 | 1360 | 0.3879 | | 0.3678 | 2.57 | 1370 | 0.3809 | | 0.3311 | 2.59 | 1380 | 0.3825 | | 0.3791 | 2.61 | 1390 | 0.3843 | | 0.3399 | 2.63 | 1400 | 0.3778 | | 0.3797 | 2.65 | 1410 | 0.3762 | | 0.357 | 2.67 | 1420 | 0.3689 | | 0.3713 | 2.69 | 1430 | 0.3724 | | 0.4323 | 2.7 | 1440 | 0.3760 | | 0.3695 | 2.72 | 1450 | 0.3782 | | 0.3662 | 2.74 | 1460 | 0.3764 | | 0.3835 | 2.76 | 1470 | 0.3815 | | 0.3748 | 2.78 | 1480 | 0.3880 | | 0.4334 | 2.8 | 1490 | 0.3935 | | 0.3937 | 2.82 | 1500 | 0.3935 | | 0.3863 | 2.84 | 1510 | 0.3840 | | 0.398 | 2.85 | 1520 | 0.3850 | | 0.4008 | 2.87 | 1530 | 0.3850 | | 0.4114 | 2.89 | 1540 | 0.3848 | | 0.4229 | 2.91 | 1550 | 0.3807 | | 0.3742 | 2.93 | 1560 | 0.3765 | | 0.3563 | 2.95 | 1570 | 0.3776 | | 0.3984 | 2.97 | 1580 | 0.3738 | | 0.3757 | 2.99 | 1590 | 0.3757 | | 0.3972 | 3.0 | 1600 | 0.3756 | | 0.3504 | 3.02 | 1610 | 0.3762 | | 0.3342 | 3.04 | 1620 | 0.3753 | | 0.3545 | 3.06 | 1630 | 0.3686 | | 0.3437 | 3.08 | 1640 | 0.3676 | | 0.3405 | 3.1 | 1650 | 0.3715 | | 0.3857 | 3.12 | 1660 | 0.3795 | | 0.3641 | 3.14 | 1670 | 0.3848 | | 0.3332 | 3.15 | 1680 | 0.3840 | | 0.3677 | 3.17 | 1690 | 0.3816 | | 0.3663 | 3.19 | 1700 | 0.3796 | | 0.3522 | 3.21 | 1710 | 0.3754 | | 0.3337 | 3.23 | 1720 | 0.3710 | | 0.3983 | 3.25 | 1730 | 0.3677 | | 0.3531 | 3.27 | 1740 | 0.3675 | | 0.343 | 3.29 | 1750 | 0.3702 | | 0.3892 | 3.31 | 1760 | 0.3723 | | 0.3398 | 3.32 | 1770 | 0.3779 | | 0.3424 | 3.34 | 1780 | 0.3719 | | 0.3609 | 3.36 | 1790 | 0.3732 | | 0.3777 | 3.38 | 1800 | 0.3705 | | 0.3841 | 3.4 | 1810 | 0.3713 | | 0.3483 | 3.42 | 1820 | 0.3689 | | 0.3943 | 3.44 | 1830 | 0.3682 | | 0.347 | 3.46 | 1840 | 0.3678 | | 0.3237 | 3.47 | 1850 | 0.3670 | | 0.3136 | 3.49 | 1860 | 0.3604 | | 0.384 | 3.51 | 1870 | 0.3551 | | 0.3938 | 3.53 | 1880 | 0.3546 | | 0.3404 | 3.55 | 1890 | 0.3593 | | 0.3441 | 3.57 | 1900 | 0.3561 | | 0.3439 | 3.59 | 1910 | 0.3601 | | 0.3607 | 3.61 | 1920 | 0.3615 | | 0.3428 | 3.62 | 1930 | 0.3614 | | 0.3889 | 3.64 | 1940 | 0.3604 | | 0.4003 | 3.66 | 1950 | 0.3589 | | 0.3373 | 3.68 | 1960 | 0.3580 | | 0.4362 | 3.7 | 1970 | 0.3594 | | 0.4004 | 3.72 | 1980 | 0.3633 | | 0.369 | 3.74 | 1990 | 0.3641 | | 0.3504 | 3.76 | 2000 | 0.3594 | | 0.3297 | 3.77 | 2010 | 0.3572 | | 0.4089 | 3.79 | 2020 | 0.3573 | | 0.3714 | 3.81 | 2030 | 0.3549 | | 0.2995 | 3.83 | 2040 | 0.3544 | | 0.342 | 3.85 | 2050 | 0.3493 | | 0.3506 | 3.87 | 2060 | 0.3486 | | 0.3223 | 3.89 | 2070 | 0.3491 | | 0.362 | 3.91 | 2080 | 0.3509 | | 0.3544 | 3.92 | 2090 | 0.3512 | | 0.3117 | 3.94 | 2100 | 0.3530 | | 0.3688 | 3.96 | 2110 | 0.3550 | | 0.3243 | 3.98 | 2120 | 0.3592 | | 0.3384 | 4.0 | 2130 | 0.3632 | | 0.3414 | 4.02 | 2140 | 0.3642 | | 0.304 | 4.04 | 2150 | 0.3632 | | 0.3341 | 4.06 | 2160 | 0.3630 | | 0.3615 | 4.08 | 2170 | 0.3661 | | 0.3405 | 4.09 | 2180 | 0.3673 | | 0.3409 | 4.11 | 2190 | 0.3701 | | 0.3339 | 4.13 | 2200 | 0.3747 | | 0.3522 | 4.15 | 2210 | 0.3746 | | 0.3435 | 4.17 | 2220 | 0.3732 | | 0.3407 | 4.19 | 2230 | 0.3761 | | 0.3562 | 4.21 | 2240 | 0.3820 | | 0.3512 | 4.23 | 2250 | 0.3842 | | 0.3887 | 4.24 | 2260 | 0.3821 | | 0.3143 | 4.26 | 2270 | 0.3811 | | 0.3548 | 4.28 | 2280 | 0.3805 | | 0.3372 | 4.3 | 2290 | 0.3752 | | 0.3452 | 4.32 | 2300 | 0.3692 | | 0.3596 | 4.34 | 2310 | 0.3683 | | 0.3533 | 4.36 | 2320 | 0.3693 | | 0.3158 | 4.38 | 2330 | 0.3708 | | 0.348 | 4.39 | 2340 | 0.3728 | | 0.3255 | 4.41 | 2350 | 0.3720 | | 0.3791 | 4.43 | 2360 | 0.3671 | | 0.3093 | 4.45 | 2370 | 0.3640 | | 0.3338 | 4.47 | 2380 | 0.3616 | | 0.3637 | 4.49 | 2390 | 0.3599 | | 0.3192 | 4.51 | 2400 | 0.3594 | | 0.3264 | 4.53 | 2410 | 0.3603 | | 0.3337 | 4.54 | 2420 | 0.3607 | | 0.3146 | 4.56 | 2430 | 0.3607 | | 0.3573 | 4.58 | 2440 | 0.3624 | | 0.3368 | 4.6 | 2450 | 0.3617 | | 0.3174 | 4.62 | 2460 | 0.3613 | | 0.3245 | 4.64 | 2470 | 0.3630 | | 0.355 | 4.66 | 2480 | 0.3619 | | 0.3731 | 4.68 | 2490 | 0.3637 | | 0.3409 | 4.69 | 2500 | 0.3651 | | 0.3393 | 4.71 | 2510 | 0.3657 | | 0.3461 | 4.73 | 2520 | 0.3665 | | 0.3353 | 4.75 | 2530 | 0.3693 | | 0.3623 | 4.77 | 2540 | 0.3685 | | 0.34 | 4.79 | 2550 | 0.3658 | | 0.3886 | 4.81 | 2560 | 0.3635 | | 0.3454 | 4.83 | 2570 | 0.3628 | | 0.3626 | 4.85 | 2580 | 0.3604 | | 0.3141 | 4.86 | 2590 | 0.3565 | | 0.342 | 4.88 | 2600 | 0.3539 | | 0.3384 | 4.9 | 2610 | 0.3535 | | 0.3603 | 4.92 | 2620 | 0.3527 | | 0.3148 | 4.94 | 2630 | 0.3534 | | 0.343 | 4.96 | 2640 | 0.3511 | | 0.3086 | 4.98 | 2650 | 0.3469 | | 0.3228 | 5.0 | 2660 | 0.3433 | | 0.3563 | 5.01 | 2670 | 0.3426 | | 0.3121 | 5.03 | 2680 | 0.3442 | | 0.2915 | 5.05 | 2690 | 0.3450 | | 0.3166 | 5.07 | 2700 | 0.3447 | | 0.3298 | 5.09 | 2710 | 0.3444 | | 0.3227 | 5.11 | 2720 | 0.3467 | | 0.3349 | 5.13 | 2730 | 0.3474 | | 0.3613 | 5.15 | 2740 | 0.3487 | | 0.3352 | 5.16 | 2750 | 0.3506 | | 0.294 | 5.18 | 2760 | 0.3507 | | 0.3205 | 5.2 | 2770 | 0.3509 | | 0.3298 | 5.22 | 2780 | 0.3478 | | 0.3644 | 5.24 | 2790 | 0.3450 | | 0.3233 | 5.26 | 2800 | 0.3440 | | 0.3329 | 5.28 | 2810 | 0.3455 | | 0.3583 | 5.3 | 2820 | 0.3472 | | 0.3272 | 5.31 | 2830 | 0.3502 | | 0.3069 | 5.33 | 2840 | 0.3520 | | 0.322 | 5.35 | 2850 | 0.3550 | | 0.3416 | 5.37 | 2860 | 0.3557 | | 0.3591 | 5.39 | 2870 | 0.3549 | | 0.3028 | 5.41 | 2880 | 0.3532 | | 0.3471 | 5.43 | 2890 | 0.3519 | | 0.3225 | 5.45 | 2900 | 0.3528 | | 0.3534 | 5.46 | 2910 | 0.3515 | | 0.3278 | 5.48 | 2920 | 0.3510 | | 0.3368 | 5.5 | 2930 | 0.3509 | | 0.3475 | 5.52 | 2940 | 0.3494 | | 0.3108 | 5.54 | 2950 | 0.3484 | | 0.3303 | 5.56 | 2960 | 0.3461 | | 0.3206 | 5.58 | 2970 | 0.3459 | | 0.3507 | 5.6 | 2980 | 0.3451 | | 0.2991 | 5.62 | 2990 | 0.3462 | | 0.3702 | 5.63 | 3000 | 0.3479 | | 0.3358 | 5.65 | 3010 | 0.3481 | | 0.3543 | 5.67 | 3020 | 0.3469 | | 0.3339 | 5.69 | 3030 | 0.3462 | | 0.3659 | 5.71 | 3040 | 0.3448 | | 0.3024 | 5.73 | 3050 | 0.3441 | | 0.3197 | 5.75 | 3060 | 0.3447 | | 0.2959 | 5.77 | 3070 | 0.3457 | | 0.3181 | 5.78 | 3080 | 0.3456 | | 0.3318 | 5.8 | 3090 | 0.3463 | | 0.3259 | 5.82 | 3100 | 0.3477 | | 0.3436 | 5.84 | 3110 | 0.3457 | | 0.3427 | 5.86 | 3120 | 0.3451 | | 0.3428 | 5.88 | 3130 | 0.3443 | | 0.3297 | 5.9 | 3140 | 0.3432 | | 0.3096 | 5.92 | 3150 | 0.3429 | | 0.3073 | 5.93 | 3160 | 0.3429 | | 0.3108 | 5.95 | 3170 | 0.3435 | | 0.3649 | 5.97 | 3180 | 0.3434 | | 0.3463 | 5.99 | 3190 | 0.3447 | | 0.3157 | 6.01 | 3200 | 0.3456 | | 0.3055 | 6.03 | 3210 | 0.3467 | | 0.3114 | 6.05 | 3220 | 0.3465 | | 0.3378 | 6.07 | 3230 | 0.3462 | | 0.3228 | 6.08 | 3240 | 0.3471 | | 0.3415 | 6.1 | 3250 | 0.3472 | | 0.3234 | 6.12 | 3260 | 0.3474 | | 0.3488 | 6.14 | 3270 | 0.3478 | | 0.3176 | 6.16 | 3280 | 0.3477 | | 0.3189 | 6.18 | 3290 | 0.3467 | | 0.3121 | 6.2 | 3300 | 0.3468 | | 0.3263 | 6.22 | 3310 | 0.3472 | | 0.3349 | 6.23 | 3320 | 0.3475 | | 0.3238 | 6.25 | 3330 | 0.3472 | | 0.3067 | 6.27 | 3340 | 0.3470 | | 0.3195 | 6.29 | 3350 | 0.3461 | | 0.3036 | 6.31 | 3360 | 0.3462 | | 0.3012 | 6.33 | 3370 | 0.3454 | | 0.3371 | 6.35 | 3380 | 0.3446 | | 0.3171 | 6.37 | 3390 | 0.3441 | | 0.3048 | 6.38 | 3400 | 0.3435 | | 0.3604 | 6.4 | 3410 | 0.3434 | | 0.2926 | 6.42 | 3420 | 0.3431 | | 0.3374 | 6.44 | 3430 | 0.3430 | | 0.3289 | 6.46 | 3440 | 0.3426 | | 0.3356 | 6.48 | 3450 | 0.3420 | | 0.3154 | 6.5 | 3460 | 0.3421 | | 0.3321 | 6.52 | 3470 | 0.3420 | | 0.2968 | 6.54 | 3480 | 0.3411 | | 0.334 | 6.55 | 3490 | 0.3406 | | 0.3393 | 6.57 | 3500 | 0.3403 | | 0.3343 | 6.59 | 3510 | 0.3402 | | 0.3198 | 6.61 | 3520 | 0.3402 | | 0.3098 | 6.63 | 3530 | 0.3405 | | 0.3059 | 6.65 | 3540 | 0.3407 | | 0.3167 | 6.67 | 3550 | 0.3409 | | 0.3502 | 6.69 | 3560 | 0.3412 | | 0.322 | 6.7 | 3570 | 0.3416 | | 0.2869 | 6.72 | 3580 | 0.3416 | | 0.3067 | 6.74 | 3590 | 0.3418 | | 0.3531 | 6.76 | 3600 | 0.3418 | | 0.3521 | 6.78 | 3610 | 0.3418 | | 0.3058 | 6.8 | 3620 | 0.3417 | | 0.3179 | 6.82 | 3630 | 0.3420 | | 0.3387 | 6.84 | 3640 | 0.3420 | | 0.3091 | 6.85 | 3650 | 0.3420 | | 0.2982 | 6.87 | 3660 | 0.3421 | | 0.3131 | 6.89 | 3670 | 0.3422 | | 0.303 | 6.91 | 3680 | 0.3421 | | 0.3524 | 6.93 | 3690 | 0.3421 | | 0.3487 | 6.95 | 3700 | 0.3421 | | 0.3225 | 6.97 | 3710 | 0.3421 | | 0.3379 | 6.99 | 3720 | 0.3421 | ### Framework versions - Transformers 4.29.2 - Pytorch 2.0.1+cu117 - Datasets 2.12.0 - Tokenizers 0.13.3