hsohn3/mayo-timebert-visit-uncased-wordlevel-block512-batch4-ep100

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.8536
• Epoch: 99

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Epoch
3.9508	0
3.4063	1
3.3682	2
3.3468	3
3.3330	4
3.3308	5
3.3225	6
3.3106	7
3.2518	8
3.1859	9
3.1373	10
3.0923	11
3.0390	12
2.9560	13
2.8605	14
2.7564	15
2.4969	16
2.2044	17
1.9566	18
1.7686	19
1.5995	20
1.4932	21
1.4100	22
1.3538	23
1.2973	24
1.2610	25
1.2160	26
1.1916	27
1.1607	28
1.1468	29
1.1262	30
1.1123	31
1.0942	32
1.0816	33
1.0717	34
1.0575	35
1.0503	36
1.0411	37
1.0293	38
1.0229	39
1.0139	40
1.0081	41
1.0028	42
0.9967	43
0.9906	44
0.9834	45
0.9782	46
0.9766	47
0.9676	48
0.9618	49
0.9611	50
0.9553	51
0.9504	52
0.9483	53
0.9404	54
0.9423	55
0.9361	56
0.9327	57
0.9327	58
0.9263	59
0.9275	60
0.9218	61
0.9202	62
0.9158	63
0.9152	64
0.9091	65
0.9104	66
0.9094	67
0.9087	68
0.9034	69
0.9063	70
0.8984	71
0.8966	72
0.8953	73
0.8910	74
0.8913	75
0.8887	76
0.8868	77
0.8868	78
0.8815	79
0.8821	80
0.8791	81
0.8752	82
0.8731	83
0.8779	84
0.8727	85
0.8702	86
0.8712	87
0.8689	88
0.8646	89
0.8644	90
0.8608	91
0.8643	92
0.8602	93
0.8605	94
0.8568	95
0.8567	96
0.8557	97
0.8543	98
0.8536	99

Framework versions

• Transformers 4.20.1
• TensorFlow 2.8.2
• Datasets 2.3.2
• Tokenizers 0.12.1