Edit model card

PromoGen_K562_2080Ti_restart

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0005
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 1000
  • num_epochs: 25
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.7676 0.49 2500 0.7383
0.7121 0.97 5000 0.6867
0.6914 1.46 7500 0.6705
0.6837 1.95 10000 0.6622
0.6778 2.44 12500 0.6558
0.6748 2.92 15000 0.6517
0.6676 3.41 17500 0.6433
0.6593 3.9 20000 0.6358
0.6584 4.38 22500 0.6320
0.6557 4.87 25000 0.6301
0.6523 5.36 27500 0.6257
0.6478 5.84 30000 0.6236
0.6393 6.33 32500 0.6145
0.6039 6.82 35000 0.5658
0.5616 7.31 37500 0.5376
0.5518 7.79 40000 0.5310
0.5509 8.28 42500 0.5273
0.5487 8.77 45000 0.5261
0.5479 9.25 47500 0.5249
0.546 9.74 50000 0.5242
0.5447 10.23 52500 0.5229
0.5439 10.71 55000 0.5220
0.5433 11.2 57500 0.5209
0.5394 11.69 60000 0.5162
0.5153 12.18 62500 0.4944
0.5137 12.66 65000 0.4932
0.514 13.15 67500 0.4924
0.5131 13.64 70000 0.4919
0.5104 14.12 72500 0.4914
0.5122 14.61 75000 0.4906
0.5089 15.1 77500 0.4901
0.5076 15.59 80000 0.4891
0.4986 16.07 82500 0.4721
0.4875 16.56 85000 0.4672
0.4887 17.05 87500 0.4669
0.4839 17.53 90000 0.4661
0.4849 18.02 92500 0.4654
0.4848 18.51 95000 0.4649
0.4831 18.99 97500 0.4646
0.4816 19.48 100000 0.4644
0.4808 19.97 102500 0.4637
0.4812 20.46 105000 0.4634
0.4813 20.94 107500 0.4633
0.4818 21.43 110000 0.4631
0.4813 21.92 112500 0.4629
0.4782 22.4 115000 0.4628
0.4804 22.89 117500 0.4626
0.4815 23.38 120000 0.4625
0.4812 23.87 122500 0.4625
0.4785 24.35 125000 0.4624
0.4795 24.84 127500 0.4624

Framework versions

  • Transformers 4.24.0
  • Pytorch 1.13.0
  • Datasets 2.7.0
  • Tokenizers 0.13.0.dev0
Downloads last month
5