amazon_rev_trained_bert_model

This model is a fine-tuned version of roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2738
• Accuracy: 0.7561

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	47	0.6940	0.5610
No log	2.0	94	0.7011	0.5854
No log	3.0	141	0.6529	0.6341
No log	4.0	188	0.6928	0.6341
No log	5.0	235	0.8478	0.6829
No log	6.0	282	0.7936	0.7805
No log	7.0	329	0.8676	0.7073
No log	8.0	376	1.6132	0.6341
No log	9.0	423	1.1779	0.7805
No log	10.0	470	1.7691	0.6098
0.4167	11.0	517	1.6079	0.7073
0.4167	12.0	564	1.6904	0.6829
0.4167	13.0	611	1.5713	0.7073
0.4167	14.0	658	1.5607	0.7317
0.4167	15.0	705	1.4878	0.7317
0.4167	16.0	752	1.9484	0.7317
0.4167	17.0	799	2.1518	0.6829
0.4167	18.0	846	2.1172	0.7073
0.4167	19.0	893	1.8095	0.7561
0.4167	20.0	940	2.0622	0.7073
0.4167	21.0	987	1.7298	0.7317
0.0952	22.0	1034	2.0204	0.7073
0.0952	23.0	1081	2.2068	0.6585
0.0952	24.0	1128	2.0693	0.7073
0.0952	25.0	1175	2.2335	0.7073
0.0952	26.0	1222	2.0538	0.7073
0.0952	27.0	1269	2.3762	0.7317
0.0952	28.0	1316	2.0004	0.7561
0.0952	29.0	1363	2.1527	0.7317
0.0952	30.0	1410	2.2242	0.7317
0.0952	31.0	1457	2.2871	0.7317
0.0273	32.0	1504	2.3725	0.7073
0.0273	33.0	1551	2.1663	0.7317
0.0273	34.0	1598	1.9155	0.7561
0.0273	35.0	1645	1.6245	0.7561
0.0273	36.0	1692	2.0135	0.7805
0.0273	37.0	1739	1.9043	0.8049
0.0273	38.0	1786	1.9403	0.7805
0.0273	39.0	1833	2.0846	0.7317
0.0273	40.0	1880	2.0752	0.7561
0.0273	41.0	1927	2.1185	0.7561
0.0273	42.0	1974	2.0753	0.7317
0.0184	43.0	2021	2.3339	0.7317
0.0184	44.0	2068	2.2412	0.7317
0.0184	45.0	2115	2.2969	0.7317
0.0184	46.0	2162	2.2884	0.7561
0.0184	47.0	2209	2.3740	0.7317
0.0184	48.0	2256	2.3022	0.7561
0.0184	49.0	2303	2.2757	0.7561
0.0184	50.0	2350	2.2738	0.7561

Framework versions

• Transformers 4.46.1
• Pytorch 2.5.1
• Datasets 2.19.1
• Tokenizers 0.20.1