2024-01-07_one_stage_subgraphs_entropyreg_txt_vis_conc_1_4_8_12_gate

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1235
• Accuracy: 0.735
• Exit 0 Accuracy: 0.0275
• Exit 1 Accuracy: 0.0625
• Exit 2 Accuracy: 0.07
• Exit 3 Accuracy: 0.3425
• Exit 4 Accuracy: 0.735

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy	Exit 2 Accuracy	Exit 3 Accuracy	Exit 4 Accuracy
No log	0.96	16	2.6652	0.1725	0.0575	0.0825	0.0625	0.0625	0.1725
No log	1.98	33	2.5287	0.245	0.055	0.0675	0.06	0.0625	0.245
No log	3.0	50	2.4248	0.285	0.055	0.0625	0.055	0.0675	0.285
No log	3.96	66	2.2424	0.375	0.0525	0.065	0.075	0.0625	0.375
No log	4.98	83	2.0686	0.45	0.0475	0.0625	0.08	0.0625	0.45
No log	6.0	100	1.7912	0.5775	0.0475	0.055	0.085	0.1175	0.5775
No log	6.96	116	1.5442	0.595	0.0325	0.0625	0.0825	0.18	0.595
No log	7.98	133	1.3694	0.655	0.0225	0.0775	0.075	0.195	0.655
No log	9.0	150	1.2956	0.675	0.0225	0.0575	0.0725	0.17	0.675
No log	9.96	166	1.2185	0.6875	0.025	0.055	0.08	0.185	0.6875
No log	10.98	183	1.1532	0.7125	0.025	0.0575	0.075	0.2325	0.7125
No log	12.0	200	1.1229	0.7075	0.025	0.0575	0.075	0.2075	0.7075
No log	12.96	216	1.1053	0.7225	0.03	0.06	0.0825	0.19	0.7225
No log	13.98	233	1.0948	0.725	0.0275	0.0575	0.0825	0.2175	0.725
No log	15.0	250	1.0867	0.725	0.025	0.0575	0.085	0.2025	0.725
No log	15.96	266	1.0750	0.7075	0.025	0.055	0.0725	0.2325	0.7075
No log	16.98	283	1.0745	0.725	0.0275	0.055	0.08	0.25	0.725
No log	18.0	300	1.0778	0.7275	0.025	0.065	0.0725	0.24	0.7275
No log	18.96	316	1.0497	0.725	0.025	0.0625	0.075	0.235	0.725
No log	19.98	333	1.0533	0.7375	0.035	0.06	0.08	0.2825	0.7375
No log	21.0	350	1.0391	0.7375	0.03	0.0675	0.0775	0.26	0.7375
No log	21.96	366	1.0249	0.745	0.03	0.065	0.075	0.2675	0.745
No log	22.98	383	1.0177	0.7525	0.03	0.0575	0.0825	0.27	0.7525
No log	24.0	400	1.0335	0.735	0.0275	0.0575	0.075	0.2775	0.735
No log	24.96	416	1.1162	0.7175	0.02	0.0575	0.075	0.2625	0.7175
No log	25.98	433	1.1288	0.7225	0.0225	0.06	0.07	0.28	0.7225
No log	27.0	450	1.0459	0.7525	0.02	0.0625	0.0625	0.3	0.7525
No log	27.96	466	1.0614	0.735	0.0225	0.0625	0.06	0.3025	0.735
No log	28.98	483	1.0468	0.7525	0.025	0.0625	0.0675	0.3125	0.7525
0.63	30.0	500	1.0359	0.745	0.0275	0.0625	0.065	0.3175	0.745
0.63	30.96	516	1.0496	0.7475	0.0275	0.06	0.06	0.3375	0.7475
0.63	31.98	533	1.0781	0.7425	0.025	0.06	0.06	0.315	0.7425
0.63	33.0	550	1.0752	0.7375	0.025	0.0625	0.0725	0.345	0.7375
0.63	33.96	566	1.0767	0.74	0.0225	0.0625	0.0575	0.32	0.74
0.63	34.98	583	1.0737	0.7425	0.0225	0.0625	0.0625	0.335	0.7425
0.63	36.0	600	1.0662	0.74	0.0225	0.0625	0.065	0.3325	0.74
0.63	36.96	616	1.0787	0.74	0.02	0.0625	0.06	0.3175	0.74
0.63	37.98	633	1.0870	0.7475	0.02	0.0625	0.05	0.3375	0.7475
0.63	39.0	650	1.0634	0.75	0.02	0.0625	0.06	0.33	0.75
0.63	39.96	666	1.1079	0.7325	0.02	0.0625	0.065	0.3175	0.7325
0.63	40.98	683	1.1028	0.73	0.0275	0.0625	0.0775	0.3375	0.73
0.63	42.0	700	1.0933	0.745	0.025	0.0625	0.0675	0.3475	0.745
0.63	42.96	716	1.1287	0.7325	0.0225	0.0625	0.0725	0.3375	0.7325
0.63	43.98	733	1.1139	0.75	0.0225	0.0625	0.075	0.335	0.75
0.63	45.0	750	1.1207	0.7325	0.025	0.0625	0.0725	0.315	0.7325
0.63	45.96	766	1.1227	0.7325	0.025	0.0625	0.08	0.33	0.7325
0.63	46.98	783	1.1072	0.7425	0.025	0.0625	0.0775	0.325	0.7425
0.63	48.0	800	1.1202	0.74	0.0275	0.0625	0.08	0.34	0.74
0.63	48.96	816	1.1230	0.745	0.0275	0.0625	0.0675	0.3475	0.745
0.63	49.98	833	1.1119	0.74	0.025	0.0625	0.0675	0.35	0.74
0.63	51.0	850	1.1131	0.7375	0.0275	0.0625	0.07	0.3425	0.7375
0.63	51.96	866	1.1306	0.735	0.0275	0.0625	0.07	0.335	0.735
0.63	52.98	883	1.1310	0.74	0.0275	0.0625	0.07	0.34	0.74
0.63	54.0	900	1.1307	0.735	0.0275	0.0625	0.07	0.35	0.735
0.63	54.96	916	1.1225	0.7425	0.0275	0.0625	0.07	0.34	0.7425
0.63	55.98	933	1.1255	0.735	0.0275	0.0625	0.07	0.3425	0.735
0.63	57.0	950	1.1243	0.735	0.0275	0.0625	0.07	0.3425	0.735
0.63	57.6	960	1.1235	0.735	0.0275	0.0625	0.07	0.3425	0.735

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3