EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-02_txt_vis_concat_enc_9_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2989
• Accuracy: 0.7625
• Exit 0 Accuracy: 0.0775
• Exit 1 Accuracy: 0.755

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6928	0.1225	0.0525	0.0625
No log	1.98	33	2.5722	0.23	0.035	0.0625
No log	3.0	50	2.3786	0.2925	0.0425	0.0625
No log	3.96	66	2.1686	0.39	0.05	0.0625
No log	4.98	83	1.9395	0.505	0.045	0.0625
No log	6.0	100	1.6538	0.5875	0.0475	0.0625
No log	6.96	116	1.4783	0.6425	0.0475	0.0625
No log	7.98	133	1.2744	0.7175	0.0525	0.0625
No log	9.0	150	1.1395	0.7325	0.06	0.0625
No log	9.96	166	1.0234	0.7525	0.065	0.0625
No log	10.98	183	0.9838	0.75	0.0675	0.0625
No log	12.0	200	0.9310	0.7475	0.065	0.0625
No log	12.96	216	0.9234	0.755	0.065	0.0625
No log	13.98	233	0.9256	0.7675	0.065	0.0625
No log	15.0	250	0.9318	0.7725	0.0675	0.0625
No log	15.96	266	0.9192	0.7475	0.0675	0.0875
No log	16.98	283	0.9302	0.7625	0.065	0.4775
No log	18.0	300	0.9552	0.76	0.065	0.685
No log	18.96	316	1.0063	0.775	0.065	0.715
No log	19.98	333	1.0117	0.7675	0.065	0.7425
No log	21.0	350	0.9867	0.775	0.065	0.77
No log	21.96	366	1.0445	0.7725	0.0675	0.775
No log	22.98	383	1.0835	0.765	0.07	0.7725
No log	24.0	400	1.0637	0.775	0.0725	0.7725
No log	24.96	416	1.1717	0.765	0.0725	0.75
No log	25.98	433	1.0935	0.7675	0.0675	0.77
No log	27.0	450	1.2155	0.7625	0.0675	0.7675
No log	27.96	466	1.1269	0.7675	0.0675	0.765
No log	28.98	483	1.1821	0.775	0.0675	0.755
1.4929	30.0	500	1.1562	0.7775	0.0725	0.7675
1.4929	30.96	516	1.1784	0.7625	0.07	0.7525
1.4929	31.98	533	1.1937	0.76	0.0725	0.7625
1.4929	33.0	550	1.2074	0.76	0.0775	0.7575
1.4929	33.96	566	1.2167	0.76	0.0775	0.7525
1.4929	34.98	583	1.2324	0.7575	0.0725	0.755
1.4929	36.0	600	1.2309	0.7525	0.075	0.755
1.4929	36.96	616	1.2377	0.76	0.0725	0.755
1.4929	37.98	633	1.2426	0.765	0.075	0.7525
1.4929	39.0	650	1.2471	0.76	0.075	0.7525
1.4929	39.96	666	1.2536	0.7625	0.075	0.7525
1.4929	40.98	683	1.2556	0.7575	0.0775	0.755
1.4929	42.0	700	1.2607	0.7625	0.0725	0.755
1.4929	42.96	716	1.2662	0.765	0.075	0.755
1.4929	43.98	733	1.2702	0.765	0.075	0.755
1.4929	45.0	750	1.2757	0.7675	0.075	0.7575
1.4929	45.96	766	1.2773	0.7675	0.0775	0.755
1.4929	46.98	783	1.2805	0.7675	0.0775	0.755
1.4929	48.0	800	1.2809	0.765	0.075	0.7525
1.4929	48.96	816	1.2841	0.76	0.0775	0.7525
1.4929	49.98	833	1.2902	0.76	0.075	0.7525
1.4929	51.0	850	1.2930	0.765	0.0775	0.755
1.4929	51.96	866	1.2963	0.7625	0.0775	0.7575
1.4929	52.98	883	1.2968	0.76	0.0775	0.7575
1.4929	54.0	900	1.2977	0.7625	0.0775	0.7575
1.4929	54.96	916	1.2978	0.765	0.0775	0.755
1.4929	55.98	933	1.2986	0.765	0.0775	0.755
1.4929	57.0	950	1.2988	0.765	0.0775	0.755
1.4929	57.6	960	1.2989	0.7625	0.0775	0.755

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3