EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-01_txt_vis_concat_enc_3_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4272
• Accuracy: 0.7725
• Exit 0 Accuracy: 0.0875
• Exit 1 Accuracy: 0.67

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6899	0.125	0.0425	0.0625
No log	1.98	33	2.5539	0.22	0.04	0.0625
No log	3.0	50	2.4098	0.3	0.04	0.0625
No log	3.96	66	2.1848	0.3725	0.0475	0.0625
No log	4.98	83	2.0107	0.42	0.0525	0.0625
No log	6.0	100	1.7450	0.535	0.0525	0.0625
No log	6.96	116	1.5086	0.6225	0.055	0.0625
No log	7.98	133	1.2913	0.6975	0.06	0.0625
No log	9.0	150	1.2016	0.7025	0.06	0.0625
No log	9.96	166	1.1037	0.72	0.0625	0.0625
No log	10.98	183	1.0035	0.7575	0.065	0.0625
No log	12.0	200	0.9654	0.76	0.0675	0.0675
No log	12.96	216	0.9382	0.7425	0.07	0.07
No log	13.98	233	0.9074	0.765	0.07	0.07
No log	15.0	250	0.9307	0.7525	0.0675	0.0875
No log	15.96	266	0.9271	0.775	0.0725	0.1
No log	16.98	283	0.9797	0.77	0.07	0.13
No log	18.0	300	1.0240	0.7625	0.075	0.1575
No log	18.96	316	0.9974	0.775	0.085	0.1725
No log	19.98	333	0.9955	0.78	0.0875	0.19
No log	21.0	350	1.0893	0.7575	0.0875	0.205
No log	21.96	366	1.1595	0.7675	0.0875	0.25
No log	22.98	383	1.0936	0.7675	0.0875	0.29
No log	24.0	400	1.0542	0.7925	0.09	0.345
No log	24.96	416	1.1406	0.7575	0.0925	0.405
No log	25.98	433	1.1886	0.77	0.09	0.4125
No log	27.0	450	1.1897	0.7625	0.09	0.4225
No log	27.96	466	1.1580	0.77	0.09	0.465
No log	28.98	483	1.1763	0.7575	0.09	0.475
1.7192	30.0	500	1.2296	0.7475	0.095	0.51
1.7192	30.96	516	1.2817	0.745	0.095	0.545
1.7192	31.98	533	1.3392	0.7425	0.0975	0.57
1.7192	33.0	550	1.4009	0.7325	0.095	0.585
1.7192	33.96	566	1.3323	0.7525	0.095	0.6075
1.7192	34.98	583	1.3872	0.7325	0.095	0.6
1.7192	36.0	600	1.3066	0.765	0.09	0.605
1.7192	36.96	616	1.3310	0.7575	0.0875	0.6075
1.7192	37.98	633	1.3706	0.75	0.085	0.615
1.7192	39.0	650	1.3312	0.765	0.09	0.62
1.7192	39.96	666	1.3910	0.765	0.09	0.63
1.7192	40.98	683	1.3507	0.77	0.09	0.655
1.7192	42.0	700	1.4006	0.7675	0.085	0.645
1.7192	42.96	716	1.4359	0.7575	0.09	0.635
1.7192	43.98	733	1.3571	0.765	0.09	0.65
1.7192	45.0	750	1.3804	0.7575	0.0875	0.6575
1.7192	45.96	766	1.4094	0.755	0.09	0.6575
1.7192	46.98	783	1.4265	0.7725	0.09	0.6525
1.7192	48.0	800	1.4834	0.74	0.085	0.66
1.7192	48.96	816	1.4232	0.76	0.085	0.6625
1.7192	49.98	833	1.4098	0.7625	0.085	0.665
1.7192	51.0	850	1.4260	0.7575	0.0875	0.66
1.7192	51.96	866	1.4142	0.77	0.0875	0.6725
1.7192	52.98	883	1.4261	0.765	0.0875	0.6625
1.7192	54.0	900	1.4103	0.775	0.0875	0.665
1.7192	54.96	916	1.4154	0.775	0.0875	0.665
1.7192	55.98	933	1.4302	0.77	0.0875	0.6675
1.7192	57.0	950	1.4274	0.7725	0.0875	0.67
1.7192	57.6	960	1.4272	0.7725	0.0875	0.67

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3