hoangtran0308/my_awesome_qa_model

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0727
• Validation Loss: 0.0552
• Epoch: 26

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 1250, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
0.0707	0.0552	0
0.0714	0.0552	1
0.0717	0.0552	2
0.0705	0.0552	3
0.0697	0.0552	4
0.0725	0.0552	5
0.0728	0.0552	6
0.0713	0.0552	7
0.0725	0.0552	8
0.0705	0.0552	9
0.0724	0.0552	10
0.0701	0.0552	11
0.0710	0.0552	12
0.0714	0.0552	13
0.0730	0.0552	14
0.0710	0.0552	15
0.0710	0.0552	16
0.0728	0.0552	17
0.0693	0.0552	18
0.0730	0.0552	19
0.0730	0.0552	20
0.0729	0.0552	21
0.0726	0.0552	22
0.0713	0.0552	23
0.0722	0.0552	24
0.0722	0.0552	25
0.0727	0.0552	26

Framework versions

• Transformers 4.34.0
• TensorFlow 2.13.0
• Datasets 2.14.5
• Tokenizers 0.14.1