Keisyahsq/NCBI_BERT

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0008
• Validation Loss: 0.1101
• Train Precision: 0.8330
• Train Recall: 0.8763
• Train F1: 0.8541
• Train Accuracy: 0.9819
• Epoch: 99

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 6790, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Train Precision	Train Recall	Train F1	Train Accuracy	Epoch
0.1192	0.0643	0.8090	0.8441	0.8261	0.9800	0
0.0402	0.0645	0.8409	0.8691	0.8547	0.9820	1
0.0227	0.0737	0.8362	0.8763	0.8558	0.9825	2
0.0117	0.0778	0.8353	0.8908	0.8621	0.9824	3
0.0067	0.0918	0.8310	0.8796	0.8546	0.9820	4
0.0044	0.0995	0.8316	0.8737	0.8521	0.9815	5
0.0030	0.1087	0.8370	0.8579	0.8473	0.9814	6
0.0021	0.1069	0.8258	0.8796	0.8519	0.9819	7
0.0013	0.1093	0.8362	0.8730	0.8542	0.9820	8
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	9
0.0013	0.1101	0.8330	0.8763	0.8541	0.9819	10
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	11
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	12
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	13
0.0011	0.1101	0.8330	0.8763	0.8541	0.9819	14
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	15
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	16
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	17
0.0007	0.1101	0.8330	0.8763	0.8541	0.9819	18
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	19
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	20
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	21
0.0007	0.1101	0.8330	0.8763	0.8541	0.9819	22
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	23
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	24
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	25
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	26
0.0007	0.1101	0.8330	0.8763	0.8541	0.9819	27
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	28
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	29
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	30
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	31
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	32
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	33
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	34
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	35
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	36
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	37
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	38
0.0011	0.1101	0.8330	0.8763	0.8541	0.9819	39
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	40
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	41
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	42
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	43
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	44
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	45
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	46
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	47
0.0011	0.1101	0.8330	0.8763	0.8541	0.9819	48
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	49
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	50
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	51
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	52
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	53
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	54
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	55
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	56
0.0011	0.1101	0.8330	0.8763	0.8541	0.9819	57
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	58
0.0007	0.1101	0.8330	0.8763	0.8541	0.9819	59
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	60
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	61
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	62
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	63
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	64
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	65
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	66
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	67
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	68
0.0011	0.1101	0.8330	0.8763	0.8541	0.9819	69
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	70
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	71
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	72
0.0007	0.1101	0.8330	0.8763	0.8541	0.9819	73
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	74
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	75
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	76
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	77
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	78
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	79
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	80
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	81
0.0007	0.1101	0.8330	0.8763	0.8541	0.9819	82
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	83
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	84
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	85
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	86
0.0007	0.1101	0.8330	0.8763	0.8541	0.9819	87
0.0010	0.1101	0.8330	0.8763	0.8541	0.9819	88
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	89
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	90
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	91
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	92
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	93
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	94
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	95
0.0009	0.1101	0.8330	0.8763	0.8541	0.9819	96
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	97
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	98
0.0008	0.1101	0.8330	0.8763	0.8541	0.9819	99

Framework versions

• Transformers 4.31.0
• TensorFlow 2.10.1
• Datasets 3.0.0
• Tokenizers 0.13.3