--- tags: - generated_from_trainer model-index: - name: PromoGen_K562_GPT2_4096_tokens_2080Ti_x4_more_DE results: [] --- # PromoGen_K562_GPT2_4096_tokens_2080Ti_x4_more_DE This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset. It achieves the following results on the evaluation set: - Loss: 10.7123 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.001 - train_batch_size: 128 - eval_batch_size: 128 - seed: 42 - gradient_accumulation_steps: 8 - total_train_batch_size: 1024 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 1000 - num_epochs: 200 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:-----:|:---------------:| | 7.8346 | 1.69 | 250 | 7.3201 | | 6.8483 | 3.38 | 500 | 6.6602 | | 6.5149 | 5.07 | 750 | 6.3857 | | 6.1941 | 6.76 | 1000 | 6.1764 | | 6.0188 | 8.45 | 1250 | 6.0308 | | 5.7091 | 10.14 | 1500 | 5.9934 | | 5.4459 | 11.82 | 1750 | 5.9371 | | 5.1559 | 13.51 | 2000 | 5.9561 | | 5.053 | 15.2 | 2250 | 6.0799 | | 4.8356 | 16.89 | 2500 | 5.9637 | | 4.6625 | 18.58 | 2750 | 6.0990 | | 4.4349 | 20.27 | 3000 | 6.4003 | | 4.2347 | 21.96 | 3250 | 6.2304 | | 4.0581 | 23.65 | 3500 | 6.4391 | | 4.1369 | 25.34 | 3750 | 6.6182 | | 3.9847 | 27.03 | 4000 | 6.8092 | | 3.7499 | 28.72 | 4250 | 6.7152 | | 3.6274 | 30.41 | 4500 | 6.8939 | | 3.515 | 32.09 | 4750 | 7.1118 | | 3.428 | 33.78 | 5000 | 6.8671 | | 3.4504 | 35.47 | 5250 | 7.0974 | | 3.3346 | 37.16 | 5500 | 7.4627 | | 3.1574 | 38.85 | 5750 | 7.1213 | | 3.0594 | 40.54 | 6000 | 7.3364 | | 3.1296 | 42.23 | 6250 | 7.5662 | | 3.0683 | 43.92 | 6500 | 7.3176 | | 2.9588 | 45.61 | 6750 | 7.4911 | | 2.8816 | 47.3 | 7000 | 7.7823 | | 2.7364 | 48.99 | 7250 | 7.4892 | | 2.6647 | 50.68 | 7500 | 7.7526 | | 2.8314 | 52.36 | 7750 | 7.9454 | | 2.7594 | 54.05 | 8000 | 8.2020 | | 2.5894 | 55.74 | 8250 | 7.8243 | | 2.5204 | 57.43 | 8500 | 8.0420 | | 2.5072 | 59.12 | 8750 | 8.2496 | | 2.4765 | 60.81 | 9000 | 7.9321 | | 2.5039 | 62.5 | 9250 | 8.1675 | | 2.4301 | 64.19 | 9500 | 8.4031 | | 2.2991 | 65.88 | 9750 | 8.0673 | | 2.2471 | 67.57 | 10000 | 8.2341 | | 2.3514 | 69.26 | 10250 | 8.5114 | | 2.3166 | 70.95 | 10500 | 8.1837 | | 2.2225 | 72.63 | 10750 | 8.3777 | | 2.17 | 74.32 | 11000 | 8.5542 | | 2.0509 | 76.01 | 11250 | 8.6312 | | 2.0204 | 77.7 | 11500 | 8.4680 | | 2.1625 | 79.39 | 11750 | 8.7197 | | 2.1102 | 81.08 | 12000 | 8.8730 | | 1.9783 | 82.77 | 12250 | 8.6112 | | 1.9293 | 84.46 | 12500 | 8.7798 | | 1.9402 | 86.15 | 12750 | 8.9528 | | 1.9095 | 87.84 | 13000 | 8.6741 | | 1.9188 | 89.53 | 13250 | 8.9483 | | 1.877 | 91.22 | 13500 | 9.0630 | | 1.7614 | 92.9 | 13750 | 8.8313 | | 1.7251 | 94.59 | 14000 | 8.9837 | | 1.821 | 96.28 | 14250 | 9.1340 | | 1.799 | 97.97 | 14500 | 8.9597 | | 1.7024 | 99.66 | 14750 | 9.1133 | | 1.6657 | 101.35 | 15000 | 9.2959 | | 1.5863 | 103.04 | 15250 | 9.2909 | | 1.5708 | 104.73 | 15500 | 9.1864 | | 1.6538 | 106.42 | 15750 | 9.3570 | | 1.6119 | 108.11 | 16000 | 9.4937 | | 1.5147 | 109.8 | 16250 | 9.3262 | | 1.4745 | 111.49 | 16500 | 9.4693 | | 1.4869 | 113.18 | 16750 | 9.5963 | | 1.4664 | 114.86 | 17000 | 9.4436 | | 1.4566 | 116.55 | 17250 | 9.5559 | | 1.4231 | 118.24 | 17500 | 9.6734 | | 1.3421 | 119.93 | 17750 | 9.5746 | | 1.3086 | 121.62 | 18000 | 9.7034 | | 1.3773 | 123.31 | 18250 | 9.7930 | | 1.3537 | 125.0 | 18500 | 9.6953 | | 1.2834 | 126.69 | 18750 | 9.8154 | | 1.2516 | 128.38 | 19000 | 9.8966 | | 1.2079 | 130.07 | 19250 | 9.9296 | | 1.1875 | 131.76 | 19500 | 9.9139 | | 1.2227 | 133.45 | 19750 | 10.0024 | | 1.1899 | 135.14 | 20000 | 10.0485 | | 1.1332 | 136.82 | 20250 | 10.0329 | | 1.107 | 138.51 | 20500 | 10.1044 | | 1.1128 | 140.2 | 20750 | 10.1534 | | 1.0945 | 141.89 | 21000 | 10.1375 | | 1.0762 | 143.58 | 21250 | 10.2039 | | 1.0495 | 145.27 | 21500 | 10.2378 | | 1.0105 | 146.96 | 21750 | 10.2377 | | 0.9875 | 148.65 | 22000 | 10.3128 | | 1.0177 | 150.34 | 22250 | 10.3502 | | 0.9956 | 152.03 | 22500 | 10.3521 | | 0.9628 | 153.72 | 22750 | 10.3924 | | 0.9403 | 155.41 | 23000 | 10.4238 | | 0.9214 | 157.09 | 23250 | 10.4514 | | 0.9078 | 158.78 | 23500 | 10.4660 | | 0.9114 | 160.47 | 23750 | 10.4969 | | 0.8935 | 162.16 | 24000 | 10.5225 | | 0.8711 | 163.85 | 24250 | 10.5399 | | 0.8549 | 165.54 | 24500 | 10.5598 | | 0.8564 | 167.23 | 24750 | 10.5896 | | 0.8441 | 168.92 | 25000 | 10.5900 | | 0.8357 | 170.61 | 25250 | 10.6085 | | 0.8229 | 172.3 | 25500 | 10.6296 | | 0.8094 | 173.99 | 25750 | 10.6426 | | 0.7992 | 175.68 | 26000 | 10.6518 | | 0.8038 | 177.36 | 26250 | 10.6652 | | 0.7948 | 179.05 | 26500 | 10.6742 | | 0.7852 | 180.74 | 26750 | 10.6780 | | 0.7775 | 182.43 | 27000 | 10.6912 | | 0.7737 | 184.12 | 27250 | 10.6925 | | 0.769 | 185.81 | 27500 | 10.6993 | | 0.7679 | 187.5 | 27750 | 10.7020 | | 0.7636 | 189.19 | 28000 | 10.7062 | | 0.76 | 190.88 | 28250 | 10.7085 | | 0.7595 | 192.57 | 28500 | 10.7107 | | 0.7583 | 194.26 | 28750 | 10.7120 | | 0.7552 | 195.95 | 29000 | 10.7126 | | 0.7568 | 197.63 | 29250 | 10.7120 | | 0.7556 | 199.32 | 29500 | 10.7123 | ### Framework versions - Transformers 4.24.0 - Pytorch 1.13.0 - Datasets 2.7.0 - Tokenizers 0.13.0.dev0