EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-08-30_txt_vis_concat_enc_1_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3611
• Accuracy: 0.755
• Exit 0 Accuracy: 0.0925
• Exit 1 Accuracy: 0.545

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.7004	0.125	0.045	0.0625
No log	1.98	33	2.5536	0.2275	0.0475	0.0575
No log	3.0	50	2.3739	0.3225	0.055	0.0675
No log	3.96	66	2.1714	0.3975	0.0625	0.1325
No log	4.98	83	1.9722	0.46	0.0675	0.135
No log	6.0	100	1.7050	0.5625	0.07	0.1225
No log	6.96	116	1.4837	0.62	0.085	0.1925
No log	7.98	133	1.3380	0.675	0.0825	0.2175
No log	9.0	150	1.2165	0.6775	0.09	0.2675
No log	9.96	166	1.1170	0.695	0.09	0.335
No log	10.98	183	1.0124	0.73	0.095	0.34
No log	12.0	200	0.9469	0.75	0.0925	0.36
No log	12.96	216	0.9763	0.715	0.0975	0.385
No log	13.98	233	0.9486	0.7425	0.09	0.41
No log	15.0	250	0.9872	0.725	0.0925	0.405
No log	15.96	266	0.9373	0.745	0.0925	0.4225
No log	16.98	283	0.9517	0.75	0.085	0.4025
No log	18.0	300	0.9673	0.745	0.08	0.4475
No log	18.96	316	1.0213	0.7425	0.0825	0.4525
No log	19.98	333	1.0320	0.75	0.085	0.44
No log	21.0	350	1.0484	0.76	0.09	0.4575
No log	21.96	366	1.0196	0.765	0.0875	0.47
No log	22.98	383	1.0498	0.75	0.085	0.45
No log	24.0	400	1.1435	0.74	0.0875	0.4775
No log	24.96	416	1.0839	0.7725	0.0875	0.4725
No log	25.98	433	1.1052	0.76	0.09	0.475
No log	27.0	450	1.1310	0.765	0.09	0.485
No log	27.96	466	1.1441	0.7775	0.09	0.4775
No log	28.98	483	1.1635	0.77	0.09	0.475
1.6676	30.0	500	1.2218	0.75	0.085	0.4825
1.6676	30.96	516	1.2521	0.76	0.09	0.495
1.6676	31.98	533	1.2850	0.7425	0.09	0.495
1.6676	33.0	550	1.2516	0.76	0.09	0.4975
1.6676	33.96	566	1.3101	0.7575	0.09	0.4975
1.6676	34.98	583	1.2974	0.765	0.09	0.5
1.6676	36.0	600	1.2978	0.7575	0.0925	0.5025
1.6676	36.96	616	1.2925	0.76	0.09	0.51
1.6676	37.98	633	1.3120	0.7625	0.0925	0.5125
1.6676	39.0	650	1.3167	0.7575	0.09	0.5125
1.6676	39.96	666	1.3262	0.7525	0.09	0.5175
1.6676	40.98	683	1.3281	0.7475	0.0925	0.5225
1.6676	42.0	700	1.3403	0.75	0.0925	0.5125
1.6676	42.96	716	1.3291	0.745	0.0925	0.5175
1.6676	43.98	733	1.3549	0.76	0.0925	0.515
1.6676	45.0	750	1.3520	0.7525	0.0925	0.5275
1.6676	45.96	766	1.3458	0.745	0.0925	0.525
1.6676	46.98	783	1.3457	0.7425	0.0925	0.535
1.6676	48.0	800	1.3441	0.7525	0.0925	0.5425
1.6676	48.96	816	1.3477	0.755	0.0925	0.5325
1.6676	49.98	833	1.3557	0.7575	0.0925	0.54
1.6676	51.0	850	1.3631	0.755	0.0925	0.54
1.6676	51.96	866	1.3643	0.7525	0.0925	0.5375
1.6676	52.98	883	1.3590	0.755	0.0925	0.5425
1.6676	54.0	900	1.3604	0.755	0.0925	0.5475
1.6676	54.96	916	1.3607	0.755	0.0925	0.5475
1.6676	55.98	933	1.3609	0.7525	0.0925	0.54
1.6676	57.0	950	1.3610	0.755	0.0925	0.545
1.6676	57.6	960	1.3611	0.755	0.0925	0.545

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3