gjonesQ02's picture
End of training
4cd033a verified
metadata
license: apache-2.0
base_model: distilgpt2
tags:
  - generated_from_trainer
model-index:
  - name: S1_InstructionGeneratorGamma
    results: []

S1_InstructionGeneratorGamma

This model is a fine-tuned version of distilgpt2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0763

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 48
  • eval_batch_size: 48
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 24

Training results

Training Loss Epoch Step Validation Loss
No log 1.0 230 0.0822
No log 2.0 460 0.0809
0.0914 3.0 690 0.0801
0.0914 4.0 920 0.0788
0.0865 5.0 1150 0.0785
0.0865 6.0 1380 0.0780
0.0841 7.0 1610 0.0781
0.0841 8.0 1840 0.0778
0.0824 9.0 2070 0.0774
0.0824 10.0 2300 0.0770
0.0817 11.0 2530 0.0771
0.0817 12.0 2760 0.0769
0.0817 13.0 2990 0.0769
0.0806 14.0 3220 0.0766
0.0806 15.0 3450 0.0765
0.0799 16.0 3680 0.0765
0.0799 17.0 3910 0.0765
0.0793 18.0 4140 0.0764
0.0793 19.0 4370 0.0763
0.0789 20.0 4600 0.0763
0.0789 21.0 4830 0.0762
0.0786 22.0 5060 0.0763
0.0786 23.0 5290 0.0763
0.0784 24.0 5520 0.0763

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2