20230823013619

This model is a fine-tuned version of bert-large-cased on the super_glue dataset. It achieves the following results on the evaluation set:

• Loss: 0.0007
• Accuracy: 0.4729

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 11
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	156	0.0076	0.5199
No log	2.0	312	0.0418	0.5343
No log	3.0	468	0.0044	0.5054
0.0669	4.0	624	0.0117	0.4693
0.0669	5.0	780	0.0333	0.4729
0.0669	6.0	936	0.0014	0.4693
0.0209	7.0	1092	0.0008	0.4729
0.0209	8.0	1248	0.0031	0.4729
0.0209	9.0	1404	0.0049	0.4982
0.0144	10.0	1560	0.0007	0.4729
0.0144	11.0	1716	0.0014	0.4693
0.0144	12.0	1872	0.0022	0.5054
0.0094	13.0	2028	0.0008	0.4729
0.0094	14.0	2184	0.0012	0.4729
0.0094	15.0	2340	0.0018	0.4729
0.0094	16.0	2496	0.0008	0.4729
0.0087	17.0	2652	0.0011	0.4729
0.0087	18.0	2808	0.0009	0.4729
0.0087	19.0	2964	0.0010	0.4729
0.0091	20.0	3120	0.0021	0.4585
0.0091	21.0	3276	0.0008	0.4729
0.0091	22.0	3432	0.0010	0.4729
0.0087	23.0	3588	0.0007	0.4729
0.0087	24.0	3744	0.0012	0.4765
0.0087	25.0	3900	0.0013	0.4729
0.0088	26.0	4056	0.0010	0.4910
0.0088	27.0	4212	0.0012	0.4765
0.0088	28.0	4368	0.0012	0.4729
0.0087	29.0	4524	0.0013	0.4910
0.0087	30.0	4680	0.0009	0.4729
0.0087	31.0	4836	0.0012	0.4729
0.0087	32.0	4992	0.0007	0.4729
0.0089	33.0	5148	0.0009	0.4729
0.0089	34.0	5304	0.0008	0.4729
0.0089	35.0	5460	0.0007	0.4729
0.0087	36.0	5616	0.0009	0.4729
0.0087	37.0	5772	0.0007	0.4801
0.0087	38.0	5928	0.0007	0.4729
0.0093	39.0	6084	0.0007	0.4729
0.0093	40.0	6240	0.0008	0.4729
0.0093	41.0	6396	0.0011	0.4729
0.0086	42.0	6552	0.0008	0.4729
0.0086	43.0	6708	0.0017	0.4729
0.0086	44.0	6864	0.0009	0.4729
0.0085	45.0	7020	0.0007	0.4729
0.0085	46.0	7176	0.0022	0.4729
0.0085	47.0	7332	0.0009	0.4729
0.0085	48.0	7488	0.0008	0.4729
0.0087	49.0	7644	0.0007	0.4729
0.0087	50.0	7800	0.0010	0.4729
0.0087	51.0	7956	0.0007	0.4729
0.0084	52.0	8112	0.0013	0.4729
0.0084	53.0	8268	0.0010	0.4729
0.0084	54.0	8424	0.0010	0.4729
0.0083	55.0	8580	0.0007	0.4729
0.0083	56.0	8736	0.0007	0.4729
0.0083	57.0	8892	0.0007	0.4729
0.0082	58.0	9048	0.0007	0.4729
0.0082	59.0	9204	0.0007	0.4729
0.0082	60.0	9360	0.0007	0.4729

Framework versions

• Transformers 4.26.1
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3