bert-nwpredict_v2

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0025
• Epoch: 49

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 1e-04, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Epoch
0.4967	0
0.2057	1
0.1540	2
0.1104	3
0.0743	4
0.0479	5
0.0293	6
0.0186	7
0.0127	8
0.0140	9
0.0087	10
0.0064	11
0.0053	12
0.0049	13
0.0040	14
0.0043	15
0.0040	16
0.0037	17
0.0040	18
0.0036	19
0.0037	20
0.0040	21
0.0038	22
0.0039	23
0.0046	24
0.0046	25
0.0053	26
0.0049	27
0.0044	28
0.0040	29
0.0034	30
0.0034	31
0.0036	32
0.0029	33
0.0028	34
0.0029	35
0.0035	36
0.0030	37
0.0027	38
0.0036	39
0.0038	40
0.0047	41
0.0037	42
0.0034	43
0.0026	44
0.0025	45
0.0024	46
0.0027	47
0.0027	48
0.0025	49

Framework versions

• Transformers 4.39.3
• TensorFlow 2.15.0
• Datasets 2.18.0
• Tokenizers 0.15.2