EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-03_txt_vis_concat_enc_11_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2071
• Accuracy: 0.7775
• Exit 0 Accuracy: 0.0775
• Exit 1 Accuracy: 0.7875

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6858	0.14	0.05	0.0625
No log	1.98	33	2.5653	0.205	0.0375	0.0625
No log	3.0	50	2.4051	0.2975	0.0475	0.0625
No log	3.96	66	2.1820	0.3775	0.04	0.0625
No log	4.98	83	1.9696	0.4775	0.0375	0.0625
No log	6.0	100	1.6723	0.555	0.0375	0.0625
No log	6.96	116	1.4666	0.605	0.0425	0.0625
No log	7.98	133	1.2438	0.6875	0.0425	0.0625
No log	9.0	150	1.1572	0.7225	0.05	0.0625
No log	9.96	166	1.0338	0.735	0.0525	0.0625
No log	10.98	183	1.0103	0.72	0.0525	0.14
No log	12.0	200	0.9362	0.7475	0.05	0.5575
No log	12.96	216	0.9409	0.74	0.055	0.69
No log	13.98	233	0.9171	0.7475	0.06	0.7575
No log	15.0	250	0.8957	0.7725	0.0625	0.7725
No log	15.96	266	0.8671	0.7775	0.0625	0.7675
No log	16.98	283	0.9132	0.7575	0.07	0.765
No log	18.0	300	0.8811	0.7725	0.0675	0.775
No log	18.96	316	0.9927	0.7625	0.0675	0.7575
No log	19.98	333	0.9015	0.7825	0.0675	0.7775
No log	21.0	350	0.9798	0.7925	0.0675	0.7775
No log	21.96	366	0.9711	0.7925	0.07	0.795
No log	22.98	383	1.0647	0.7725	0.07	0.7775
No log	24.0	400	1.0429	0.765	0.0725	0.775
No log	24.96	416	1.0613	0.775	0.075	0.7775
No log	25.98	433	1.0366	0.78	0.0725	0.7925
No log	27.0	450	1.0424	0.7725	0.07	0.78
No log	27.96	466	1.0550	0.7775	0.0675	0.7825
No log	28.98	483	1.0691	0.7775	0.07	0.785
1.3822	30.0	500	1.0771	0.78	0.075	0.7775
1.3822	30.96	516	1.0844	0.78	0.07	0.7825
1.3822	31.98	533	1.0930	0.7775	0.075	0.785
1.3822	33.0	550	1.1125	0.78	0.0775	0.7825
1.3822	33.96	566	1.1169	0.785	0.075	0.785
1.3822	34.98	583	1.1258	0.7825	0.0725	0.7825
1.3822	36.0	600	1.1369	0.78	0.0725	0.7825
1.3822	36.96	616	1.1400	0.78	0.0725	0.785
1.3822	37.98	633	1.1484	0.78	0.0725	0.785
1.3822	39.0	650	1.1513	0.7825	0.0725	0.7825
1.3822	39.96	666	1.1561	0.78	0.075	0.7875
1.3822	40.98	683	1.1555	0.785	0.075	0.785
1.3822	42.0	700	1.1595	0.7825	0.0725	0.785
1.3822	42.96	716	1.1675	0.7775	0.075	0.78
1.3822	43.98	733	1.1744	0.7775	0.0725	0.785
1.3822	45.0	750	1.1780	0.7775	0.075	0.7875
1.3822	45.96	766	1.1841	0.7775	0.075	0.7875
1.3822	46.98	783	1.1896	0.7775	0.0775	0.7875
1.3822	48.0	800	1.1891	0.7775	0.075	0.7825
1.3822	48.96	816	1.1911	0.7775	0.0775	0.785
1.3822	49.98	833	1.1937	0.7775	0.075	0.785
1.3822	51.0	850	1.1964	0.7775	0.075	0.785
1.3822	51.96	866	1.2002	0.7775	0.0775	0.785
1.3822	52.98	883	1.2016	0.7775	0.0775	0.785
1.3822	54.0	900	1.2035	0.775	0.0775	0.785
1.3822	54.96	916	1.2052	0.7775	0.0775	0.7875
1.3822	55.98	933	1.2069	0.78	0.0775	0.7875
1.3822	57.0	950	1.2071	0.7775	0.0775	0.7875
1.3822	57.6	960	1.2071	0.7775	0.0775	0.7875

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3