Hugging Face's logo

kasrahabib
/
gpt2-finetuned-iso29148-req-detector

Text Classification
Transformers
TensorFlow
TensorBoard
gpt2
generated_from_keras_callback
Inference Endpoints
text-generation-inference
Model card Files Metrics Community
gpt2-finetuned-iso29148-req-detector / README.md
kasrahabib's picture
kasrahabib
Upload folder using huggingface_hub
338204e verified
preview code
|
raw history blame contribute delete
No virus
2.96 kB
metadata
license: mit
base_model: openai-community/gpt2
tags:
  - generated_from_keras_callback
model-index:
  - name: kasrahabib/gpt2-finetuned-iso29148-req-detector
    results: []

kasrahabib/gpt2-finetuned-iso29148-req-detector

This model is a fine-tuned version of openai-community/gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 0.1004
  • Validation Loss: 0.5468
  • Epoch: 29

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 3570, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
  • training_precision: float32

Training results

Train Loss Validation Loss Epoch
3.0225 2.8144 0
2.7666 2.7501 1
2.6971 2.6517 2
2.6218 2.4804 3
2.3698 2.0353 4
1.8887 1.4317 5
1.1104 0.8991 6
0.7810 0.8057 7
0.6463 0.6800 8
0.5507 0.6747 9
0.4679 0.6156 10
0.4313 0.6102 11
0.3696 0.5972 12
0.3365 0.5437 13
0.2866 0.5637 14
0.2492 0.5682 15
0.2392 0.5496 16
0.2029 0.6076 17
0.2062 0.5368 18
0.1898 0.5434 19
0.1670 0.5434 20
0.1682 0.5403 21
0.1650 0.5830 22
0.1326 0.5400 23
0.1366 0.5497 24
0.1148 0.5432 25
0.1263 0.5662 26
0.1171 0.5591 27
0.1075 0.5445 28
0.1004 0.5468 29

Framework versions

  • Transformers 4.40.1
  • TensorFlow 2.15.0
  • Datasets 2.19.1
  • Tokenizers 0.19.1