roberta-base-sst-2-32-13

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9957
• Accuracy: 0.8438

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.6951	0.5
No log	2.0	4	0.6951	0.5
No log	3.0	6	0.6951	0.5
No log	4.0	8	0.6951	0.5
0.6937	5.0	10	0.6950	0.5
0.6937	6.0	12	0.6950	0.5
0.6937	7.0	14	0.6950	0.5
0.6937	8.0	16	0.6950	0.5
0.6937	9.0	18	0.6949	0.5
0.6953	10.0	20	0.6949	0.5
0.6953	11.0	22	0.6949	0.5
0.6953	12.0	24	0.6948	0.5
0.6953	13.0	26	0.6948	0.5
0.6953	14.0	28	0.6947	0.5
0.6975	15.0	30	0.6947	0.5
0.6975	16.0	32	0.6946	0.5
0.6975	17.0	34	0.6946	0.5
0.6975	18.0	36	0.6945	0.5
0.6975	19.0	38	0.6944	0.5
0.6888	20.0	40	0.6944	0.5
0.6888	21.0	42	0.6943	0.5
0.6888	22.0	44	0.6942	0.5
0.6888	23.0	46	0.6942	0.5
0.6888	24.0	48	0.6941	0.5
0.6947	25.0	50	0.6940	0.5
0.6947	26.0	52	0.6940	0.5
0.6947	27.0	54	0.6939	0.5
0.6947	28.0	56	0.6938	0.5
0.6947	29.0	58	0.6937	0.5
0.69	30.0	60	0.6937	0.5
0.69	31.0	62	0.6936	0.5
0.69	32.0	64	0.6936	0.5
0.69	33.0	66	0.6935	0.5
0.69	34.0	68	0.6934	0.5
0.6901	35.0	70	0.6933	0.5
0.6901	36.0	72	0.6932	0.5
0.6901	37.0	74	0.6931	0.5
0.6901	38.0	76	0.6930	0.5
0.6901	39.0	78	0.6929	0.5
0.6895	40.0	80	0.6928	0.5
0.6895	41.0	82	0.6927	0.5
0.6895	42.0	84	0.6926	0.5
0.6895	43.0	86	0.6925	0.5
0.6895	44.0	88	0.6924	0.5
0.6874	45.0	90	0.6922	0.5
0.6874	46.0	92	0.6921	0.5
0.6874	47.0	94	0.6919	0.5
0.6874	48.0	96	0.6917	0.5
0.6874	49.0	98	0.6915	0.5
0.6865	50.0	100	0.6913	0.5
0.6865	51.0	102	0.6911	0.5
0.6865	52.0	104	0.6908	0.5
0.6865	53.0	106	0.6904	0.4844
0.6865	54.0	108	0.6901	0.4688
0.6818	55.0	110	0.6897	0.4688
0.6818	56.0	112	0.6892	0.4531
0.6818	57.0	114	0.6887	0.5625
0.6818	58.0	116	0.6880	0.6094
0.6818	59.0	118	0.6872	0.6406
0.6697	60.0	120	0.6863	0.6406
0.6697	61.0	122	0.6852	0.6875
0.6697	62.0	124	0.6838	0.7656
0.6697	63.0	126	0.6820	0.7812
0.6697	64.0	128	0.6798	0.7656
0.6559	65.0	130	0.6769	0.7656
0.6559	66.0	132	0.6730	0.7188
0.6559	67.0	134	0.6675	0.7344
0.6559	68.0	136	0.6598	0.7188
0.6559	69.0	138	0.6489	0.7188
0.6085	70.0	140	0.6343	0.7188
0.6085	71.0	142	0.6161	0.7656
0.6085	72.0	144	0.5928	0.8125
0.6085	73.0	146	0.5652	0.8438
0.6085	74.0	148	0.5367	0.8594
0.474	75.0	150	0.5083	0.8438
0.474	76.0	152	0.4779	0.8438
0.474	77.0	154	0.4473	0.8594
0.474	78.0	156	0.4179	0.8594
0.474	79.0	158	0.3930	0.875
0.2428	80.0	160	0.3782	0.8594
0.2428	81.0	162	0.3734	0.8438
0.2428	82.0	164	0.3731	0.8594
0.2428	83.0	166	0.3816	0.875
0.2428	84.0	168	0.4042	0.8438
0.0805	85.0	170	0.4405	0.8438
0.0805	86.0	172	0.4840	0.8281
0.0805	87.0	174	0.5432	0.8125
0.0805	88.0	176	0.6025	0.8125
0.0805	89.0	178	0.6412	0.8125
0.0222	90.0	180	0.6653	0.8125
0.0222	91.0	182	0.6845	0.8125
0.0222	92.0	184	0.6954	0.8125
0.0222	93.0	186	0.7007	0.8281
0.0222	94.0	188	0.7029	0.8438
0.0093	95.0	190	0.7083	0.8438
0.0093	96.0	192	0.7172	0.8594
0.0093	97.0	194	0.7250	0.8594
0.0093	98.0	196	0.7286	0.8594
0.0093	99.0	198	0.7361	0.8594
0.0058	100.0	200	0.7447	0.8594
0.0058	101.0	202	0.7544	0.8594
0.0058	102.0	204	0.7632	0.8594
0.0058	103.0	206	0.7724	0.8594
0.0058	104.0	208	0.7842	0.8594
0.0041	105.0	210	0.7955	0.8594
0.0041	106.0	212	0.8061	0.8594
0.0041	107.0	214	0.8164	0.8594
0.0041	108.0	216	0.8262	0.8594
0.0041	109.0	218	0.8348	0.8594
0.0032	110.0	220	0.8438	0.8594
0.0032	111.0	222	0.8514	0.8594
0.0032	112.0	224	0.8582	0.8594
0.0032	113.0	226	0.8650	0.8594
0.0032	114.0	228	0.8718	0.8438
0.0028	115.0	230	0.8777	0.8438
0.0028	116.0	232	0.8829	0.8438
0.0028	117.0	234	0.8884	0.8438
0.0028	118.0	236	0.8938	0.8438
0.0028	119.0	238	0.8986	0.8438
0.0024	120.0	240	0.9023	0.8438
0.0024	121.0	242	0.9055	0.8438
0.0024	122.0	244	0.9087	0.8438
0.0024	123.0	246	0.9121	0.8438
0.0024	124.0	248	0.9165	0.8438
0.0021	125.0	250	0.9209	0.8438
0.0021	126.0	252	0.9258	0.8438
0.0021	127.0	254	0.9303	0.8438
0.0021	128.0	256	0.9338	0.8438
0.0021	129.0	258	0.9365	0.8438
0.0019	130.0	260	0.9395	0.8438
0.0019	131.0	262	0.9426	0.8438
0.0019	132.0	264	0.9448	0.8438
0.0019	133.0	266	0.9463	0.8438
0.0019	134.0	268	0.9480	0.8438
0.0017	135.0	270	0.9506	0.8438
0.0017	136.0	272	0.9535	0.8438
0.0017	137.0	274	0.9561	0.8438
0.0017	138.0	276	0.9579	0.8438
0.0017	139.0	278	0.9596	0.8438
0.0015	140.0	280	0.9618	0.8438
0.0015	141.0	282	0.9650	0.8438
0.0015	142.0	284	0.9682	0.8438
0.0015	143.0	286	0.9712	0.8438
0.0015	144.0	288	0.9741	0.8438
0.0014	145.0	290	0.9769	0.8438
0.0014	146.0	292	0.9801	0.8438
0.0014	147.0	294	0.9835	0.8438
0.0014	148.0	296	0.9872	0.8438
0.0014	149.0	298	0.9911	0.8438
0.0013	150.0	300	0.9957	0.8438

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3