Keisyahsq/JNLPBA_BERT

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0319
• Validation Loss: 0.5084
• Train Precision: 0.7117
• Train Recall: 0.8098
• Train F1: 0.7576
• Train Accuracy: 0.9046
• Epoch: 81

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'inner_optimizer': {'class_name': 'AdamWeightDecay', 'config': {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 23180, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}}, 'dynamic': True, 'initial_scale': 32768.0, 'dynamic_growth_steps': 2000}
• training_precision: mixed_float16

Training results

Train Loss	Validation Loss	Train Precision	Train Recall	Train F1	Train Accuracy	Epoch
0.0322	0.5084	0.7115	0.8098	0.7575	0.9046	0
0.0319	0.5084	0.7115	0.8098	0.7575	0.9046	1
0.0323	0.5084	0.7115	0.8098	0.7575	0.9046	2
0.0317	0.5084	0.7115	0.8098	0.7575	0.9046	3
0.0317	0.5084	0.7115	0.8098	0.7575	0.9046	4
0.0318	0.5084	0.7115	0.8098	0.7575	0.9046	5
0.0318	0.5084	0.7115	0.8098	0.7575	0.9046	6
0.0318	0.5084	0.7116	0.8098	0.7575	0.9046	7
0.0317	0.5084	0.7116	0.8098	0.7575	0.9046	8
0.0324	0.5084	0.7116	0.8098	0.7575	0.9046	9
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	10
0.0321	0.5084	0.7116	0.8098	0.7575	0.9046	11
0.0318	0.5084	0.7116	0.8098	0.7575	0.9046	12
0.0323	0.5084	0.7116	0.8098	0.7575	0.9046	13
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	14
0.0322	0.5084	0.7116	0.8098	0.7575	0.9046	15
0.0319	0.5084	0.7116	0.8098	0.7575	0.9046	16
0.0325	0.5084	0.7116	0.8098	0.7575	0.9046	17
0.0325	0.5084	0.7116	0.8098	0.7575	0.9046	18
0.0323	0.5084	0.7116	0.8098	0.7575	0.9046	19
0.0326	0.5084	0.7116	0.8098	0.7575	0.9046	20
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	21
0.0325	0.5084	0.7116	0.8098	0.7575	0.9046	22
0.0319	0.5084	0.7116	0.8098	0.7575	0.9046	23
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	24
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	25
0.0322	0.5084	0.7116	0.8098	0.7575	0.9046	26
0.0317	0.5084	0.7116	0.8098	0.7575	0.9046	27
0.0322	0.5084	0.7116	0.8098	0.7575	0.9046	28
0.0322	0.5084	0.7116	0.8098	0.7575	0.9046	29
0.0321	0.5084	0.7116	0.8098	0.7575	0.9046	30
0.0322	0.5084	0.7116	0.8098	0.7575	0.9046	31
0.0318	0.5084	0.7116	0.8098	0.7575	0.9046	32
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	33
0.0318	0.5084	0.7116	0.8098	0.7575	0.9046	34
0.0318	0.5084	0.7116	0.8098	0.7575	0.9046	35
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	36
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	37
0.0320	0.5084	0.7116	0.8098	0.7575	0.9046	38
0.0319	0.5084	0.7116	0.8098	0.7575	0.9046	39
0.0321	0.5084	0.7116	0.8098	0.7575	0.9046	40
0.0325	0.5084	0.7116	0.8098	0.7575	0.9046	41
0.0319	0.5084	0.7116	0.8098	0.7575	0.9046	42
0.0315	0.5084	0.7117	0.8098	0.7576	0.9046	43
0.0321	0.5084	0.7117	0.8098	0.7576	0.9046	44
0.0322	0.5084	0.7117	0.8098	0.7576	0.9046	45
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	46
0.0319	0.5084	0.7117	0.8098	0.7576	0.9046	47
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	48
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	49
0.0321	0.5084	0.7117	0.8098	0.7576	0.9046	50
0.0320	0.5084	0.7117	0.8098	0.7576	0.9046	51
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	52
0.0321	0.5084	0.7117	0.8098	0.7576	0.9046	53
0.0322	0.5084	0.7117	0.8098	0.7576	0.9046	54
0.0320	0.5084	0.7117	0.8098	0.7576	0.9046	55
0.0321	0.5084	0.7117	0.8098	0.7576	0.9046	56
0.0322	0.5084	0.7117	0.8099	0.7576	0.9047	57
0.0319	0.5084	0.7117	0.8098	0.7576	0.9046	58
0.0321	0.5084	0.7117	0.8099	0.7576	0.9047	59
0.0320	0.5084	0.7117	0.8099	0.7576	0.9046	60
0.0320	0.5084	0.7117	0.8099	0.7576	0.9047	61
0.0320	0.5084	0.7117	0.8098	0.7576	0.9046	62
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	63
0.0316	0.5084	0.7117	0.8098	0.7576	0.9046	64
0.0319	0.5084	0.7117	0.8098	0.7576	0.9046	65
0.0317	0.5084	0.7117	0.8098	0.7576	0.9046	66
0.0320	0.5084	0.7117	0.8098	0.7576	0.9046	67
0.0317	0.5084	0.7117	0.8098	0.7576	0.9046	68
0.0326	0.5084	0.7117	0.8099	0.7576	0.9046	69
0.0321	0.5084	0.7117	0.8098	0.7576	0.9046	70
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	71
0.0320	0.5084	0.7117	0.8098	0.7576	0.9046	72
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	73
0.0316	0.5084	0.7116	0.8098	0.7576	0.9046	74
0.0324	0.5084	0.7117	0.8099	0.7576	0.9046	75
0.0319	0.5084	0.7117	0.8099	0.7576	0.9046	76
0.0322	0.5084	0.7117	0.8099	0.7576	0.9046	77
0.0318	0.5084	0.7117	0.8098	0.7576	0.9046	78
0.0316	0.5084	0.7117	0.8098	0.7576	0.9046	79
0.0316	0.5084	0.7117	0.8098	0.7576	0.9046	80
0.0319	0.5084	0.7117	0.8098	0.7576	0.9046	81

Framework versions

• Transformers 4.31.0
• TensorFlow 2.10.1
• Datasets 3.0.0
• Tokenizers 0.13.3