2024-01-09_one_stage_subgraphs_weighted_entropyreg_txt_vis_conc_2_5_9_11_gate

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9390
• Accuracy: 0.77
• Exit 0 Accuracy: 0.0625
• Exit 1 Accuracy: 0.0625
• Exit 2 Accuracy: 0.06
• Exit 3 Accuracy: 0.0625
• Exit 4 Accuracy: 0.74

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy	Exit 2 Accuracy	Exit 3 Accuracy	Exit 4 Accuracy
No log	0.96	16	2.6696	0.195	0.065	0.0625	0.0625	0.0625	0.0625
No log	1.98	33	2.5026	0.2575	0.0675	0.0625	0.0625	0.0625	0.0625
No log	3.0	50	2.3137	0.31	0.0675	0.0625	0.0625	0.0625	0.0625
No log	3.96	66	2.0882	0.4325	0.07	0.0625	0.0625	0.0625	0.0625
No log	4.98	83	1.8166	0.56	0.07	0.0625	0.0625	0.0625	0.0625
No log	6.0	100	1.5497	0.645	0.0725	0.0625	0.0625	0.0625	0.0625
No log	6.96	116	1.3812	0.655	0.0725	0.0625	0.0625	0.0625	0.0725
No log	7.98	133	1.2204	0.7175	0.0725	0.0625	0.0625	0.0625	0.085
No log	9.0	150	1.1322	0.75	0.0725	0.0625	0.0625	0.0625	0.095
No log	9.96	166	1.0710	0.755	0.0725	0.0625	0.0625	0.0625	0.0925
No log	10.98	183	1.0184	0.7525	0.0675	0.0625	0.0625	0.0625	0.115
No log	12.0	200	1.0033	0.74	0.0725	0.0625	0.055	0.0625	0.1275
No log	12.96	216	0.9933	0.745	0.0725	0.0625	0.06	0.0625	0.1525
No log	13.98	233	0.9915	0.735	0.07	0.0625	0.0625	0.0625	0.18
No log	15.0	250	0.9726	0.7525	0.07	0.0625	0.0625	0.0625	0.18
No log	15.96	266	0.9361	0.765	0.07	0.0625	0.0625	0.0625	0.205
No log	16.98	283	0.8964	0.78	0.065	0.0625	0.0625	0.0625	0.2225
No log	18.0	300	0.9296	0.7575	0.065	0.0625	0.0625	0.0625	0.2525
No log	18.96	316	0.9034	0.7725	0.0625	0.0625	0.06	0.0625	0.255
No log	19.98	333	0.9168	0.7675	0.065	0.0625	0.0625	0.0625	0.2975
No log	21.0	350	0.8820	0.78	0.0625	0.0625	0.0625	0.0625	0.3825
No log	21.96	366	0.9458	0.7625	0.065	0.0625	0.0625	0.0625	0.4375
No log	22.98	383	0.9000	0.775	0.065	0.0625	0.0625	0.0625	0.4675
No log	24.0	400	0.9042	0.785	0.0625	0.0625	0.0625	0.0625	0.48
No log	24.96	416	0.8947	0.795	0.065	0.0625	0.0625	0.0625	0.505
No log	25.98	433	0.9094	0.7925	0.065	0.0625	0.065	0.0625	0.52
No log	27.0	450	0.9078	0.785	0.065	0.0625	0.0625	0.0625	0.555
No log	27.96	466	0.9380	0.78	0.065	0.0625	0.0625	0.0625	0.57
No log	28.98	483	0.9616	0.765	0.065	0.0625	0.0625	0.0625	0.5775
0.4296	30.0	500	0.9114	0.785	0.065	0.0625	0.0625	0.0625	0.575
0.4296	30.96	516	0.9216	0.7825	0.065	0.0625	0.0625	0.0625	0.575
0.4296	31.98	533	0.9074	0.7875	0.065	0.0625	0.0625	0.0625	0.6
0.4296	33.0	550	0.9205	0.785	0.065	0.0625	0.0625	0.0625	0.5975
0.4296	33.96	566	0.9177	0.79	0.065	0.0625	0.0625	0.0625	0.6175
0.4296	34.98	583	0.9161	0.785	0.065	0.0625	0.0625	0.0625	0.64
0.4296	36.0	600	0.9309	0.7775	0.065	0.0625	0.0625	0.0625	0.65
0.4296	36.96	616	0.9311	0.7725	0.065	0.0625	0.0625	0.0625	0.6525
0.4296	37.98	633	0.9310	0.77	0.065	0.0625	0.0625	0.0625	0.65
0.4296	39.0	650	0.9295	0.7725	0.065	0.0625	0.0625	0.0625	0.6625
0.4296	39.96	666	0.9194	0.78	0.0625	0.0625	0.0625	0.0625	0.6775
0.4296	40.98	683	0.9221	0.775	0.0625	0.0625	0.0625	0.0625	0.69
0.4296	42.0	700	0.9324	0.7725	0.0625	0.0625	0.0625	0.0625	0.695
0.4296	42.96	716	0.9245	0.775	0.0625	0.0625	0.065	0.0625	0.6975
0.4296	43.98	733	0.9348	0.7725	0.0625	0.0625	0.0625	0.0625	0.7175
0.4296	45.0	750	0.9295	0.775	0.0625	0.0625	0.0625	0.0625	0.72
0.4296	45.96	766	0.9334	0.77	0.0625	0.0625	0.0625	0.0625	0.7225
0.4296	46.98	783	0.9358	0.775	0.0625	0.0625	0.0625	0.0625	0.725
0.4296	48.0	800	0.9280	0.7775	0.0625	0.0625	0.0625	0.0625	0.735
0.4296	48.96	816	0.9403	0.77	0.0625	0.0625	0.0625	0.0625	0.735
0.4296	49.98	833	0.9304	0.7725	0.0625	0.0625	0.0625	0.0625	0.7325
0.4296	51.0	850	0.9356	0.7725	0.0625	0.0625	0.0625	0.0625	0.7325
0.4296	51.96	866	0.9392	0.7725	0.0625	0.0625	0.06	0.0625	0.7375
0.4296	52.98	883	0.9370	0.7725	0.0625	0.0625	0.06	0.0625	0.74
0.4296	54.0	900	0.9404	0.7675	0.0625	0.0625	0.06	0.0625	0.745
0.4296	54.96	916	0.9405	0.7725	0.0625	0.0625	0.06	0.0625	0.7425
0.4296	55.98	933	0.9409	0.77	0.0625	0.0625	0.06	0.0625	0.74
0.4296	57.0	950	0.9391	0.77	0.0625	0.0625	0.06	0.0625	0.74
0.4296	57.6	960	0.9390	0.77	0.0625	0.0625	0.06	0.0625	0.74

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3