bert-large-cased-sigir-support-no-label-40

This model is a fine-tuned version of bert-large-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1107

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 30
• eval_batch_size: 30
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.7638	1.0	246	2.2805
2.1924	2.0	492	1.9602
1.8921	3.0	738	1.7992
1.7412	4.0	984	1.7229
1.6311	5.0	1230	1.6165
1.5421	6.0	1476	1.5400
1.4619	7.0	1722	1.5001
1.3846	8.0	1968	1.4381
1.3414	9.0	2214	1.4285
1.2894	10.0	2460	1.4108
1.2467	11.0	2706	1.3460
1.1992	12.0	2952	1.3434
1.1612	13.0	3198	1.2951
1.1266	14.0	3444	1.2518
1.0933	15.0	3690	1.2825
1.0625	16.0	3936	1.2523
1.0386	17.0	4182	1.2251
1.0066	18.0	4428	1.2339
0.9755	19.0	4674	1.1887
0.9656	20.0	4920	1.2288
0.9517	21.0	5166	1.1391
0.9207	22.0	5412	1.1718
0.8964	23.0	5658	1.1850
0.8891	24.0	5904	1.1306
0.8564	25.0	6150	1.1956
0.851	26.0	6396	1.1263
0.8331	27.0	6642	1.1060
0.8143	28.0	6888	1.0689
0.7972	29.0	7134	1.0772
0.7857	30.0	7380	1.1103
0.7687	31.0	7626	1.1635
0.7653	32.0	7872	1.0736
0.777	33.0	8118	1.1103
0.741	34.0	8364	1.0830
0.7408	35.0	8610	1.0809
0.736	36.0	8856	1.0894
0.7362	37.0	9102	1.0691
0.727	38.0	9348	1.0519
0.715	39.0	9594	1.0919
0.7286	40.0	9840	1.1107

Framework versions

• Transformers 4.26.0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2