---
license: mit
base_model: openai-community/gpt2
tags:
- trl
- sft
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: '47015772_1'
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 47015772_1

This model is a fine-tuned version of [openai-community/gpt2](https://huggingface.co/openai-community/gpt2) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6501
- Accuracy: 0.0001

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1.41e-05
- train_batch_size: 32
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- gradient_accumulation_steps: 4
- total_train_batch_size: 256
- total_eval_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2000

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 1.1019        | 0.03  | 25   | 0.9747          | 0.0001   |
| 0.8784        | 0.06  | 50   | 0.8178          | 0.0001   |
| 0.8308        | 0.09  | 75   | 0.7727          | 0.0002   |
| 0.7975        | 0.11  | 100  | 0.7508          | 0.0001   |
| 0.787         | 0.14  | 125  | 0.7370          | 0.0001   |
| 0.7752        | 0.17  | 150  | 0.7279          | 0.0001   |
| 0.7703        | 0.2   | 175  | 0.7210          | 0.0001   |
| 0.7584        | 0.23  | 200  | 0.7157          | 0.0001   |
| 0.7662        | 0.26  | 225  | 0.7108          | 0.0001   |
| 0.7431        | 0.28  | 250  | 0.7073          | 0.0001   |
| 0.7437        | 0.31  | 275  | 0.7038          | 0.0001   |
| 0.7378        | 0.34  | 300  | 0.7006          | 0.0001   |
| 0.7252        | 0.37  | 325  | 0.6974          | 0.0001   |
| 0.7242        | 0.4   | 350  | 0.6948          | 0.0001   |
| 0.7284        | 0.43  | 375  | 0.6922          | 0.0001   |
| 0.7187        | 0.46  | 400  | 0.6898          | 0.0001   |
| 0.7169        | 0.48  | 425  | 0.6878          | 0.0001   |
| 0.723         | 0.51  | 450  | 0.6858          | 0.0001   |
| 0.7189        | 0.54  | 475  | 0.6834          | 0.0001   |
| 0.7142        | 0.57  | 500  | 0.6818          | 0.0001   |
| 0.7071        | 0.6   | 525  | 0.6798          | 0.0001   |
| 0.7131        | 0.63  | 550  | 0.6782          | 0.0001   |
| 0.7107        | 0.66  | 575  | 0.6768          | 0.0001   |
| 0.702         | 0.68  | 600  | 0.6761          | 0.0001   |
| 0.6937        | 0.71  | 625  | 0.6742          | 0.0001   |
| 0.6984        | 0.74  | 650  | 0.6732          | 0.0001   |
| 0.7023        | 0.77  | 675  | 0.6720          | 0.0001   |
| 0.7002        | 0.8   | 700  | 0.6708          | 0.0001   |
| 0.7057        | 0.83  | 725  | 0.6697          | 0.0001   |
| 0.6991        | 0.85  | 750  | 0.6687          | 0.0001   |
| 0.6951        | 0.88  | 775  | 0.6674          | 0.0001   |
| 0.7101        | 0.91  | 800  | 0.6668          | 0.0001   |
| 0.6931        | 0.94  | 825  | 0.6661          | 0.0001   |
| 0.6866        | 0.97  | 850  | 0.6652          | 0.0001   |
| 0.702         | 1.0   | 875  | 0.6643          | 0.0001   |
| 0.6996        | 1.03  | 900  | 0.6637          | 0.0001   |
| 0.7077        | 1.05  | 925  | 0.6629          | 0.0001   |
| 0.6955        | 1.08  | 950  | 0.6623          | 0.0001   |
| 0.6947        | 1.11  | 975  | 0.6619          | 0.0001   |
| 0.6933        | 1.14  | 1000 | 0.6612          | 0.0001   |
| 0.6855        | 1.17  | 1025 | 0.6601          | 0.0001   |
| 0.7004        | 1.2   | 1050 | 0.6597          | 0.0001   |
| 0.6934        | 1.22  | 1075 | 0.6594          | 0.0001   |
| 0.6863        | 1.25  | 1100 | 0.6585          | 0.0001   |
| 0.6886        | 1.28  | 1125 | 0.6580          | 0.0001   |
| 0.6851        | 1.31  | 1150 | 0.6579          | 0.0001   |
| 0.683         | 1.34  | 1175 | 0.6574          | 0.0001   |
| 0.703         | 1.37  | 1200 | 0.6570          | 0.0001   |
| 0.6792        | 1.4   | 1225 | 0.6564          | 0.0001   |
| 0.6849        | 1.42  | 1250 | 0.6558          | 0.0001   |
| 0.6856        | 1.45  | 1275 | 0.6556          | 0.0001   |
| 0.6856        | 1.48  | 1300 | 0.6551          | 0.0001   |
| 0.6856        | 1.51  | 1325 | 0.6550          | 0.0001   |
| 0.6857        | 1.54  | 1350 | 0.6543          | 0.0001   |
| 0.6856        | 1.57  | 1375 | 0.6539          | 0.0001   |
| 0.689         | 1.59  | 1400 | 0.6538          | 0.0001   |
| 0.6892        | 1.62  | 1425 | 0.6534          | 0.0001   |
| 0.6823        | 1.65  | 1450 | 0.6532          | 0.0001   |
| 0.6828        | 1.68  | 1475 | 0.6529          | 0.0001   |
| 0.6864        | 1.71  | 1500 | 0.6528          | 0.0001   |
| 0.6886        | 1.74  | 1525 | 0.6523          | 0.0001   |
| 0.6642        | 1.77  | 1550 | 0.6521          | 0.0001   |
| 0.6849        | 1.79  | 1575 | 0.6519          | 0.0001   |
| 0.6834        | 1.82  | 1600 | 0.6516          | 0.0001   |
| 0.6839        | 1.85  | 1625 | 0.6515          | 0.0001   |
| 0.6856        | 1.88  | 1650 | 0.6514          | 0.0001   |
| 0.6725        | 1.91  | 1675 | 0.6511          | 0.0001   |
| 0.6813        | 1.94  | 1700 | 0.6509          | 0.0001   |
| 0.6832        | 1.97  | 1725 | 0.6508          | 0.0001   |
| 0.6739        | 1.99  | 1750 | 0.6508          | 0.0001   |
| 0.6716        | 2.02  | 1775 | 0.6506          | 0.0001   |
| 0.6798        | 2.05  | 1800 | 0.6505          | 0.0001   |
| 0.6758        | 2.08  | 1825 | 0.6503          | 0.0001   |
| 0.6791        | 2.11  | 1850 | 0.6503          | 0.0001   |
| 0.6735        | 2.14  | 1875 | 0.6503          | 0.0001   |
| 0.6887        | 2.16  | 1900 | 0.6502          | 0.0001   |
| 0.686         | 2.19  | 1925 | 0.6502          | 0.0001   |
| 0.682         | 2.22  | 1950 | 0.6501          | 0.0001   |
| 0.675         | 2.25  | 1975 | 0.6501          | 0.0001   |
| 0.6792        | 2.28  | 2000 | 0.6501          | 0.0001   |


### Framework versions

- Transformers 4.37.0
- Pytorch 2.0.0+cu118
- Datasets 2.16.1
- Tokenizers 0.15.1