EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-08-24_txt_vis_concat_enc_1_2_3_4_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5649
• Accuracy: 0.7425
• Exit 0 Accuracy: 0.05
• Exit 1 Accuracy: 0.415
• Exit 2 Accuracy: 0.505
• Exit 3 Accuracy: 0.6
• Exit 4 Accuracy: 0.6425

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy	Exit 2 Accuracy	Exit 3 Accuracy	Exit 4 Accuracy
No log	0.96	16	2.6888	0.15	0.08	0.0625	0.0625	0.0625	0.0625
No log	1.98	33	2.5275	0.225	0.08	0.105	0.0625	0.0625	0.0625
No log	3.0	50	2.3637	0.3025	0.0775	0.11	0.0625	0.0625	0.0625
No log	3.96	66	2.1405	0.38	0.075	0.1275	0.0625	0.0625	0.0625
No log	4.98	83	1.8868	0.4975	0.075	0.11	0.0625	0.0625	0.0625
No log	6.0	100	1.6298	0.59	0.075	0.12	0.0625	0.0625	0.0625
No log	6.96	116	1.4167	0.64	0.0725	0.1225	0.0625	0.0625	0.0625
No log	7.98	133	1.2772	0.67	0.075	0.1225	0.0625	0.0625	0.0625
No log	9.0	150	1.1184	0.7325	0.075	0.125	0.0625	0.0625	0.0625
No log	9.96	166	1.0215	0.7275	0.07	0.1225	0.0625	0.0625	0.07
No log	10.98	183	0.9752	0.7525	0.07	0.12	0.0625	0.0625	0.07
No log	12.0	200	0.9165	0.7425	0.07	0.13	0.0625	0.0625	0.0775
No log	12.96	216	0.9352	0.7475	0.0725	0.1325	0.0625	0.0625	0.0775
No log	13.98	233	0.9210	0.745	0.0725	0.13	0.0625	0.0625	0.0775
No log	15.0	250	0.8671	0.775	0.075	0.1275	0.0625	0.0625	0.09
No log	15.96	266	0.9380	0.7625	0.0725	0.1275	0.0625	0.0625	0.095
No log	16.98	283	0.9594	0.77	0.0725	0.1225	0.0625	0.0625	0.1025
No log	18.0	300	1.0292	0.745	0.0725	0.1275	0.0625	0.0625	0.1025
No log	18.96	316	0.9903	0.755	0.07	0.13	0.08	0.0625	0.1075
No log	19.98	333	1.0235	0.7725	0.065	0.1275	0.08	0.065	0.1175
No log	21.0	350	1.0540	0.7675	0.0675	0.1175	0.09	0.0825	0.1275
No log	21.96	366	1.1432	0.745	0.075	0.1375	0.0875	0.1175	0.1825
No log	22.98	383	1.1439	0.75	0.0725	0.1575	0.0775	0.17	0.2475
No log	24.0	400	1.2294	0.7325	0.07	0.21	0.12	0.1975	0.26
No log	24.96	416	1.2759	0.73	0.07	0.1425	0.1325	0.1925	0.2725
No log	25.98	433	1.1571	0.765	0.06	0.17	0.155	0.2475	0.3275
No log	27.0	450	1.2853	0.7475	0.0575	0.205	0.185	0.275	0.3825
No log	27.96	466	1.3344	0.7325	0.0475	0.2525	0.2575	0.3275	0.3875
No log	28.98	483	1.2372	0.7475	0.06	0.2075	0.2325	0.32	0.4425
1.7096	30.0	500	1.2672	0.7625	0.0525	0.2775	0.34	0.3825	0.4775
1.7096	30.96	516	1.3086	0.7525	0.0525	0.3225	0.375	0.4425	0.51
1.7096	31.98	533	1.3129	0.7525	0.0525	0.29	0.3825	0.4175	0.525
1.7096	33.0	550	1.3782	0.735	0.0475	0.305	0.4075	0.4625	0.525
1.7096	33.96	566	1.3449	0.735	0.0475	0.33	0.425	0.48	0.5425
1.7096	34.98	583	1.4527	0.7325	0.045	0.34	0.435	0.4925	0.5475
1.7096	36.0	600	1.4438	0.7275	0.05	0.3525	0.43	0.52	0.5425
1.7096	36.96	616	1.5117	0.7275	0.045	0.3775	0.445	0.53	0.56
1.7096	37.98	633	1.4637	0.735	0.0475	0.3925	0.445	0.5425	0.5675
1.7096	39.0	650	1.5315	0.73	0.045	0.3875	0.4575	0.55	0.6
1.7096	39.96	666	1.4396	0.74	0.05	0.39	0.4625	0.555	0.5975
1.7096	40.98	683	1.4850	0.7425	0.05	0.39	0.455	0.5475	0.6025
1.7096	42.0	700	1.4815	0.7525	0.05	0.3975	0.4625	0.5675	0.6
1.7096	42.96	716	1.4511	0.7475	0.05	0.3975	0.4725	0.56	0.6175
1.7096	43.98	733	1.5443	0.7275	0.05	0.3975	0.47	0.56	0.625
1.7096	45.0	750	1.5364	0.725	0.05	0.3975	0.4825	0.5675	0.625
1.7096	45.96	766	1.5455	0.7325	0.05	0.4	0.49	0.5675	0.625
1.7096	46.98	783	1.4992	0.745	0.05	0.4	0.4875	0.58	0.62
1.7096	48.0	800	1.5089	0.7375	0.05	0.4025	0.485	0.5825	0.6325
1.7096	48.96	816	1.5149	0.7375	0.05	0.4025	0.4925	0.5875	0.63
1.7096	49.98	833	1.5285	0.735	0.05	0.4025	0.5025	0.59	0.635
1.7096	51.0	850	1.5455	0.73	0.05	0.4	0.4975	0.595	0.64
1.7096	51.96	866	1.5598	0.7425	0.05	0.42	0.5	0.5975	0.64
1.7096	52.98	883	1.5727	0.7325	0.05	0.4125	0.5	0.5925	0.64
1.7096	54.0	900	1.5694	0.7425	0.05	0.415	0.495	0.5975	0.64
1.7096	54.96	916	1.5760	0.735	0.05	0.415	0.5025	0.5975	0.64
1.7096	55.98	933	1.5687	0.74	0.05	0.4125	0.5025	0.5975	0.6425
1.7096	57.0	950	1.5648	0.74	0.05	0.415	0.5025	0.6	0.6425
1.7096	57.6	960	1.5649	0.7425	0.05	0.415	0.505	0.6	0.6425

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3