sabiq101/LEAD_B

pretrained_MaterialsBERT

This model is a fine-tuned version of pranav-s/MaterialsBERT on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2020
• Accuracy: 0.7271

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 3
• total_train_batch_size: 48
• total_eval_batch_size: 48
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.6452	0.2641	500	1.5018	0.6847
1.5379	0.5283	1000	1.4544	0.6895
1.497	0.7924	1500	1.4186	0.6957
1.4604	1.0565	2000	1.3934	0.6987
1.4366	1.3207	2500	1.3694	0.7021
1.4045	1.5848	3000	1.3524	0.7049
1.3919	1.8489	3500	1.3361	0.7068
1.3841	2.1130	4000	1.3278	0.7086
1.3711	2.3772	4500	1.3318	0.7077
1.3627	2.6413	5000	1.3148	0.7100
1.3493	2.9054	5500	1.3028	0.7122
1.3423	3.1696	6000	1.2972	0.7123
1.3327	3.4337	6500	1.2910	0.7141
1.3235	3.6978	7000	1.2826	0.7148
1.3125	3.9620	7500	1.2749	0.7160
1.3039	4.2261	8000	1.2726	0.7163
1.2998	4.4902	8500	1.2621	0.7182
1.2962	4.7544	9000	1.2672	0.7171
1.295	5.0185	9500	1.2645	0.7173
1.2868	5.2826	10000	1.2480	0.7198
1.2765	5.5468	10500	1.2482	0.7196
1.2758	5.8109	11000	1.2436	0.7205
1.2686	6.0750	11500	1.2460	0.7207
1.2589	6.3391	12000	1.2369	0.7215
1.2634	6.6033	12500	1.2372	0.7219
1.25	6.8674	13000	1.2199	0.7250
1.2387	7.1315	13500	1.2175	0.7246
1.2374	7.3957	14000	1.2169	0.7244
1.2369	7.6598	14500	1.2130	0.7254
1.2338	7.9239	15000	1.2141	0.7244
1.23	8.1881	15500	1.2124	0.7257
1.2295	8.4522	16000	1.2078	0.7259
1.2229	8.7163	16500	1.2095	0.7261
1.2254	8.9805	17000	1.2058	0.7262
1.2234	9.2446	17500	1.2024	0.7267
1.2173	9.5087	18000	1.2052	0.7272
1.2197	9.7728	18500	1.2003	0.7278

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.7.0+cu126
• Datasets 2.21.0
• Tokenizers 0.21.1