bert-large-cased-sigir-LR100-1-cased-40

This model is a fine-tuned version of bert-large-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2085

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 30
• eval_batch_size: 30
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
6.661	1.0	1	6.7088
7.1425	2.0	2	7.0634
6.8918	3.0	3	6.2872
6.1875	4.0	4	5.5826
5.6201	5.0	5	5.4365
5.181	6.0	6	3.7720
5.0548	7.0	7	5.5019
4.3957	8.0	8	3.2004
3.993	9.0	9	2.4284
3.593	10.0	10	3.2126
3.754	11.0	11	2.7146
3.061	12.0	12	2.5308
3.0496	13.0	13	2.8430
3.1128	14.0	14	1.2934
2.7098	15.0	15	1.5709
2.5303	16.0	16	1.9032
2.3475	17.0	17	2.1788
2.4054	18.0	18	1.5836
2.6168	19.0	19	3.7077
2.5972	20.0	20	2.8996
2.287	21.0	21	2.1028
2.1383	22.0	22	2.0755
2.443	23.0	23	1.6498
2.0233	24.0	24	2.2023
2.2446	25.0	25	2.4627
1.9087	26.0	26	2.3244
2.1685	27.0	27	1.9509
1.9055	28.0	28	2.6149
1.9063	29.0	29	2.0499
2.3587	30.0	30	1.1757
2.0389	31.0	31	1.1181
1.9223	32.0	32	1.6205
2.0361	33.0	33	1.8381
2.1823	34.0	34	0.7964
2.2411	35.0	35	2.0179
1.8976	36.0	36	1.1467
1.9321	37.0	37	1.5334
2.257	38.0	38	2.1575
2.0543	39.0	39	1.5084
1.7383	40.0	40	1.8176

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2