Metrics

• loss: 1.6841
• accuracy: 0.7884
• precision: 0.8028
• recall: 0.7884
• precision_macro: 0.6408
• recall_macro: 0.6436
• macro_fpr: 0.0956
• weighted_fpr: 0.0821
• weighted_specificity: 0.8781
• macro_specificity: 0.9166
• weighted_sensitivity: 0.7884
• macro_sensitivity: 0.6436
• f1_micro: 0.7884
• f1_macro: 0.6410
• f1_weighted: 0.7953
• runtime: 229.8279
• samples_per_second: 1.9540
• steps_per_second: 0.2480

case-analysis-roberta-base

This model is a fine-tuned version of FacebookAI/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6841
• Accuracy: 0.7884
• Precision: 0.8028
• Recall: 0.7884
• Precision Macro: 0.6320
• Recall Macro: 0.6238
• Macro Fpr: 0.0958
• Weighted Fpr: 0.0781
• Weighted Specificity: 0.8648
• Macro Specificity: 0.9155
• Weighted Sensitivity: 0.7973
• Macro Sensitivity: 0.6238
• F1 Micro: 0.7973
• F1 Macro: 0.6277
• F1 Weighted: 0.7968

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	Precision Macro	Recall Macro	Macro Fpr	Weighted Fpr	Weighted Specificity	Macro Specificity	Weighted Sensitivity	Macro Sensitivity	F1 Micro	F1 Macro	F1 Weighted
No log	1.0	224	0.8771	0.7528	0.7120	0.7528	0.5404	0.5402	0.1314	0.0987	0.7806	0.8833	0.7528	0.5402	0.7528	0.5389	0.7301
No log	2.0	448	0.7936	0.7728	0.7420	0.7728	0.5529	0.5937	0.1080	0.0892	0.8458	0.9046	0.7728	0.5937	0.7728	0.5712	0.7555
0.8855	3.0	672	0.8127	0.7305	0.7321	0.7305	0.5336	0.5600	0.1288	0.1095	0.8209	0.8879	0.7305	0.5600	0.7305	0.5313	0.7208
0.8855	4.0	896	1.0186	0.7795	0.7503	0.7795	0.5722	0.5654	0.1184	0.0862	0.8004	0.8950	0.7795	0.5654	0.7795	0.5605	0.7561
0.5551	5.0	1120	0.7591	0.8085	0.7674	0.8085	0.5833	0.5963	0.0988	0.0732	0.8375	0.9115	0.8085	0.5963	0.8085	0.5892	0.7867
0.5551	6.0	1344	0.9522	0.8174	0.7816	0.8174	0.6117	0.5988	0.0967	0.0693	0.8297	0.9118	0.8174	0.5988	0.8174	0.6030	0.7967
0.386	7.0	1568	1.0569	0.7706	0.7610	0.7706	0.5710	0.5858	0.1089	0.0903	0.8522	0.9057	0.7706	0.5858	0.7706	0.5782	0.7656
0.386	8.0	1792	1.1957	0.7572	0.7918	0.7572	0.6175	0.6264	0.1052	0.0965	0.8905	0.9119	0.7572	0.6264	0.7572	0.6162	0.7715
0.2709	9.0	2016	1.2092	0.7728	0.7897	0.7728	0.6331	0.6301	0.1021	0.0892	0.8751	0.9120	0.7728	0.6301	0.7728	0.6264	0.7773
0.2709	10.0	2240	1.3830	0.7706	0.7782	0.7706	0.6112	0.6073	0.1094	0.0903	0.8464	0.9043	0.7706	0.6073	0.7706	0.6072	0.7728
0.2709	11.0	2464	1.4518	0.7817	0.7944	0.7817	0.6157	0.6059	0.1030	0.0851	0.8606	0.9106	0.7817	0.6059	0.7817	0.6077	0.7856
0.1837	12.0	2688	1.5283	0.7684	0.7840	0.7684	0.6143	0.6003	0.1058	0.0913	0.8701	0.9096	0.7684	0.6003	0.7684	0.6022	0.7726
0.1837	13.0	2912	1.5136	0.7817	0.7907	0.7817	0.6231	0.6472	0.0979	0.0851	0.8733	0.9137	0.7817	0.6472	0.7817	0.6332	0.7848
0.1212	14.0	3136	1.6569	0.7506	0.8138	0.7506	0.6380	0.6499	0.1039	0.0997	0.8911	0.9104	0.7506	0.6499	0.7506	0.6327	0.7764
0.1212	15.0	3360	1.5305	0.7661	0.7714	0.7661	0.5965	0.6203	0.1054	0.0923	0.8710	0.9093	0.7661	0.6203	0.7661	0.6068	0.7669
0.0793	16.0	3584	1.4931	0.7996	0.7896	0.7996	0.6016	0.6193	0.0947	0.0771	0.8625	0.9155	0.7996	0.6193	0.7996	0.6085	0.7933
0.0793	17.0	3808	1.4582	0.8018	0.7911	0.8018	0.6143	0.6131	0.0963	0.0761	0.8523	0.9135	0.8018	0.6131	0.8018	0.6132	0.7958
0.0473	18.0	4032	1.6772	0.7795	0.7924	0.7795	0.6154	0.6342	0.0990	0.0862	0.8742	0.9134	0.7795	0.6342	0.7795	0.6224	0.7843
0.0473	19.0	4256	1.5707	0.7929	0.7890	0.7929	0.6409	0.6339	0.0966	0.0801	0.8666	0.9149	0.7929	0.6339	0.7929	0.6348	0.7892
0.0473	20.0	4480	1.4891	0.8018	0.8136	0.8018	0.6441	0.6284	0.0916	0.0761	0.8768	0.9196	0.8018	0.6284	0.8018	0.6355	0.8073
0.0476	21.0	4704	1.5064	0.8062	0.8181	0.8062	0.6511	0.6320	0.0896	0.0742	0.8754	0.9204	0.8062	0.6320	0.8062	0.6407	0.8117
0.0476	22.0	4928	1.5076	0.8107	0.8003	0.8107	0.6296	0.6247	0.0913	0.0722	0.8642	0.9187	0.8107	0.6247	0.8107	0.6265	0.8050
0.0366	23.0	5152	1.5891	0.7973	0.8113	0.7973	0.6455	0.6382	0.0929	0.0781	0.8763	0.9184	0.7973	0.6382	0.7973	0.6407	0.8038
0.0366	24.0	5376	1.6779	0.7951	0.7990	0.7951	0.6306	0.5982	0.0994	0.0791	0.8581	0.9133	0.7951	0.5982	0.7951	0.6123	0.7956
0.0368	25.0	5600	1.6211	0.8040	0.8024	0.8040	0.6420	0.6223	0.0952	0.0751	0.8570	0.9152	0.8040	0.6223	0.8040	0.6313	0.8023
0.0368	26.0	5824	1.4841	0.8062	0.8060	0.8062	0.6364	0.6416	0.0894	0.0742	0.8775	0.9209	0.8062	0.6416	0.8062	0.6385	0.8058
0.0252	27.0	6048	1.6841	0.7884	0.8028	0.7884	0.6408	0.6436	0.0956	0.0821	0.8781	0.9166	0.7884	0.6436	0.7884	0.6410	0.7953
0.0252	28.0	6272	1.7185	0.7929	0.8006	0.7929	0.6386	0.6338	0.0954	0.0801	0.8725	0.9163	0.7929	0.6338	0.7929	0.6355	0.7964
0.0252	29.0	6496	1.6500	0.7996	0.7989	0.7996	0.6338	0.6276	0.0942	0.0771	0.8678	0.9168	0.7996	0.6276	0.7996	0.6306	0.7992
0.0147	30.0	6720	1.6506	0.7973	0.7965	0.7973	0.6320	0.6238	0.0958	0.0781	0.8648	0.9155	0.7973	0.6238	0.7973	0.6277	0.7968

Framework versions

• Transformers 4.40.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1