hsohn3/cchs-bert-visit-uncased-wordlevel-block512-batch4-ep100

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.7195
• Epoch: 99

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Epoch
3.8730	0
3.0562	1
3.0168	2
3.0032	3
2.9954	4
2.9951	5
2.9904	6
2.9765	7
2.9788	8
2.9692	9
2.9656	10
2.9761	11
2.9643	12
2.9393	13
2.9026	14
2.8685	15
2.8438	16
2.8279	17
2.8107	18
2.7896	19
2.7716	20
2.7458	21
2.7118	22
2.6519	23
2.5933	24
2.4702	25
2.2842	26
2.0712	27
1.8406	28
1.6374	29
1.4836	30
1.3824	31
1.3079	32
1.2538	33
1.2054	34
1.1700	35
1.1432	36
1.1122	37
1.0939	38
1.0645	39
1.0465	40
1.0248	41
1.0069	42
0.9902	43
0.9769	44
0.9510	45
0.9394	46
0.9316	47
0.9181	48
0.9090	49
0.9010	50
0.8934	51
0.8791	52
0.8759	53
0.8652	54
0.8566	55
0.8511	56
0.8414	57
0.8373	58
0.8302	59
0.8241	60
0.8246	61
0.8207	62
0.8110	63
0.8081	64
0.8010	65
0.7995	66
0.7965	67
0.7941	68
0.7849	69
0.7866	70
0.7874	71
0.7796	72
0.7742	73
0.7706	74
0.7687	75
0.7686	76
0.7663	77
0.7586	78
0.7554	79
0.7563	80
0.7541	81
0.7527	82
0.7482	83
0.7460	84
0.7436	85
0.7423	86
0.7422	87
0.7385	88
0.7367	89
0.7321	90
0.7320	91
0.7354	92
0.7271	93
0.7270	94
0.7210	95
0.7236	96
0.7263	97
0.7237	98
0.7195	99

Framework versions

• Transformers 4.20.1
• TensorFlow 2.8.2
• Datasets 2.3.2
• Tokenizers 0.12.1