shahajbert_nwp_finetuning_test4

This model is a fine-tuned version of neuropark/sahajBERT on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.3512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	442	3.5414
3.5816	2.0	884	3.6077
3.3982	3.0	1326	3.4116
3.2805	4.0	1768	3.4087
3.1704	5.0	2210	3.2837
3.1138	6.0	2652	3.3092
3.0029	7.0	3094	3.3947
2.9647	8.0	3536	3.3681
2.9647	9.0	3978	3.3163
2.905	10.0	4420	3.3362
2.8483	11.0	4862	3.3709
2.7664	12.0	5304	3.2465
2.7149	13.0	5746	3.2881
2.7167	14.0	6188	3.2806
2.6327	15.0	6630	3.2336
2.5984	16.0	7072	3.2800
2.5561	17.0	7514	3.3218
2.5561	18.0	7956	3.2904
2.5021	19.0	8398	3.2421
2.4415	20.0	8840	3.2416
2.4228	21.0	9282	3.2291
2.3859	22.0	9724	3.2795
2.3504	23.0	10166	3.2575
2.2906	24.0	10608	3.2737
2.2716	25.0	11050	3.3618
2.2716	26.0	11492	3.2623
2.2671	27.0	11934	3.2853
2.2169	28.0	12376	3.3880
2.2031	29.0	12818	3.4195
2.1467	30.0	13260	3.3009
2.139	31.0	13702	3.3603
2.1133	32.0	14144	3.2665
2.0534	33.0	14586	3.4353
2.077	34.0	15028	3.3433
2.077	35.0	15470	3.3460
2.0511	36.0	15912	3.3844
2.0123	37.0	16354	3.2761
1.9915	38.0	16796	3.4099
1.975	39.0	17238	3.3159
1.9539	40.0	17680	3.3269
1.9193	41.0	18122	3.3377
1.9181	42.0	18564	3.4188
1.9071	43.0	19006	3.4064
1.9071	44.0	19448	3.1825
1.8729	45.0	19890	3.3935
1.877	46.0	20332	3.3168
1.8656	47.0	20774	3.3597
1.8549	48.0	21216	3.2954
1.8614	49.0	21658	3.3801
1.8642	50.0	22100	3.3173

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3