Edit model card

ecolibrium

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 0.3202
  • Validation Loss: 0.0689
  • Epoch: 49

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 0.0002, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
  • training_precision: float32

Training results

Train Loss Validation Loss Epoch
1.6155 1.3909 0
1.4232 1.2592 1
1.3301 1.1768 2
1.2562 1.0908 3
1.1925 1.0136 4
1.1417 0.9589 5
1.0953 0.9173 6
1.0502 0.8531 7
1.0103 0.8009 8
0.9761 0.7488 9
0.9404 0.7100 10
0.9095 0.6793 11
0.8743 0.6319 12
0.8480 0.6139 13
0.8233 0.5741 14
0.7942 0.5479 15
0.7697 0.5176 16
0.7456 0.4847 17
0.7250 0.4650 18
0.6996 0.4370 19
0.6790 0.4141 20
0.6607 0.3959 21
0.6428 0.3666 22
0.6249 0.3511 23
0.6060 0.3344 24
0.5944 0.3178 25
0.5750 0.2942 26
0.5607 0.2787 27
0.5453 0.2608 28
0.5317 0.2472 29
0.5146 0.2365 30
0.5017 0.2146 31
0.4909 0.2078 32
0.4764 0.1945 33
0.4664 0.1831 34
0.4517 0.1703 35
0.4397 0.1643 36
0.4316 0.1588 37
0.4196 0.1428 38
0.4073 0.1311 39
0.3949 0.1232 40
0.3871 0.1175 41
0.3776 0.1105 42
0.3705 0.1025 43
0.3623 0.0959 44
0.3514 0.0928 45
0.3427 0.0828 46
0.3346 0.0799 47
0.3268 0.0736 48
0.3202 0.0689 49

Framework versions

  • Transformers 4.35.0
  • TensorFlow 2.14.0
  • Datasets 2.14.6
  • Tokenizers 0.14.1
Downloads last month
6

Finetuned from