metadata

license: mit
base_model: openai-community/gpt2
tags:
  - generated_from_keras_callback
model-index:
  - name: kasrahabib/gpt2-finetuned-iso29148-req-detector
    results: []

kasrahabib/gpt2-finetuned-iso29148-req-detector

This model is a fine-tuned version of openai-community/gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

Train Loss: 0.1004
Validation Loss: 0.5468
Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 3570, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
3.0225	2.8144	0
2.7666	2.7501	1
2.6971	2.6517	2
2.6218	2.4804	3
2.3698	2.0353	4
1.8887	1.4317	5
1.1104	0.8991	6
0.7810	0.8057	7
0.6463	0.6800	8
0.5507	0.6747	9
0.4679	0.6156	10
0.4313	0.6102	11
0.3696	0.5972	12
0.3365	0.5437	13
0.2866	0.5637	14
0.2492	0.5682	15
0.2392	0.5496	16
0.2029	0.6076	17
0.2062	0.5368	18
0.1898	0.5434	19
0.1670	0.5434	20
0.1682	0.5403	21
0.1650	0.5830	22
0.1326	0.5400	23
0.1366	0.5497	24
0.1148	0.5432	25
0.1263	0.5662	26
0.1171	0.5591	27
0.1075	0.5445	28
0.1004	0.5468	29

Framework versions

Transformers 4.40.1
TensorFlow 2.15.0
Datasets 2.19.1
Tokenizers 0.19.1