AbSadr-Finetuned-Transformer

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 2.5273
• Validation Loss: 2.3977
• Train Rougel: tf.Tensor(0.1196819, shape=(), dtype=float32)
• Epoch: 9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': np.float32(2e-05), 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Rougel	Epoch
3.2855	2.7337	tf.Tensor(0.10175698, shape=(), dtype=float32)	0
2.8676	2.6013	tf.Tensor(0.10433203, shape=(), dtype=float32)	1
2.7597	2.5356	tf.Tensor(0.106469, shape=(), dtype=float32)	2
2.7072	2.4959	tf.Tensor(0.108870566, shape=(), dtype=float32)	3
2.6633	2.4679	tf.Tensor(0.11151629, shape=(), dtype=float32)	4
2.6263	2.4454	tf.Tensor(0.113661006, shape=(), dtype=float32)	5
2.5999	2.4272	tf.Tensor(0.11553433, shape=(), dtype=float32)	6
2.5726	2.4139	tf.Tensor(0.117180854, shape=(), dtype=float32)	7
2.5493	2.4024	tf.Tensor(0.11846918, shape=(), dtype=float32)	8
2.5273	2.3977	tf.Tensor(0.1196819, shape=(), dtype=float32)	9

Framework versions

• Transformers 4.51.1
• TensorFlow 2.18.0
• Datasets 3.5.0
• Tokenizers 0.21.1