hsohn3/mayo-bert-visit-uncased-wordlevel-block512-batch4-ep50

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 1.0832
• Epoch: 49

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Epoch
4.1266	0
3.5212	1
3.4780	2
3.4533	3
3.4376	4
3.4325	5
3.4276	6
3.4119	7
3.3654	8
3.2948	9
3.2422	10
3.2069	11
3.1736	12
3.1292	13
3.0781	14
3.0138	15
2.9582	16
2.8954	17
2.7166	18
2.4073	19
2.1573	20
1.9469	21
1.7981	22
1.7012	23
1.6145	24
1.5482	25
1.4866	26
1.4336	27
1.3815	28
1.3483	29
1.3087	30
1.2814	31
1.2548	32
1.2373	33
1.2216	34
1.2080	35
1.1907	36
1.1820	37
1.1650	38
1.1622	39
1.1498	40
1.1347	41
1.1299	42
1.1197	43
1.1100	44
1.1094	45
1.1033	46
1.0943	47
1.0918	48
1.0832	49

Framework versions

• Transformers 4.20.1
• TensorFlow 2.8.2
• Datasets 2.3.2
• Tokenizers 0.12.1