EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-08-30_txt_vis_concat_enc_2_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2478
• Accuracy: 0.795
• Exit 0 Accuracy: 0.085
• Exit 1 Accuracy: 0.57

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6891	0.1275	0.0525	0.0625
No log	1.98	33	2.5473	0.2425	0.04	0.0625
No log	3.0	50	2.3563	0.32	0.0475	0.0625
No log	3.96	66	2.1010	0.405	0.05	0.0625
No log	4.98	83	1.8280	0.515	0.05	0.0625
No log	6.0	100	1.6080	0.5925	0.05	0.0625
No log	6.96	116	1.3814	0.66	0.0525	0.0625
No log	7.98	133	1.2221	0.715	0.0525	0.0625
No log	9.0	150	1.1049	0.7375	0.06	0.0625
No log	9.96	166	1.0433	0.71	0.0575	0.0625
No log	10.98	183	0.9452	0.7625	0.0575	0.0625
No log	12.0	200	0.9152	0.7575	0.0575	0.0625
No log	12.96	216	0.9473	0.7575	0.065	0.0625
No log	13.98	233	0.9487	0.7525	0.065	0.0625
No log	15.0	250	0.9706	0.7625	0.065	0.0625
No log	15.96	266	0.9101	0.7925	0.0775	0.0625
No log	16.98	283	0.9571	0.7725	0.065	0.0625
No log	18.0	300	1.0558	0.76	0.0675	0.0625
No log	18.96	316	0.9547	0.77	0.0675	0.1075
No log	19.98	333	1.0204	0.7575	0.07	0.18
No log	21.0	350	1.1142	0.75	0.0675	0.25
No log	21.96	366	1.1336	0.75	0.0675	0.3425
No log	22.98	383	1.0917	0.76	0.07	0.3825
No log	24.0	400	1.1059	0.765	0.075	0.4325
No log	24.96	416	1.1171	0.775	0.075	0.4025
No log	25.98	433	1.0902	0.78	0.08	0.44
No log	27.0	450	1.1270	0.785	0.08	0.465
No log	27.96	466	1.1483	0.7925	0.0775	0.465
No log	28.98	483	1.1501	0.7875	0.0825	0.4875
1.6846	30.0	500	1.2854	0.7575	0.075	0.4925
1.6846	30.96	516	1.1910	0.7775	0.0725	0.5
1.6846	31.98	533	1.2389	0.77	0.0725	0.51
1.6846	33.0	550	1.2157	0.7775	0.0775	0.52
1.6846	33.96	566	1.2510	0.7675	0.075	0.52
1.6846	34.98	583	1.2536	0.7775	0.075	0.5225
1.6846	36.0	600	1.2163	0.7825	0.0825	0.5125
1.6846	36.96	616	1.1992	0.78	0.08	0.515
1.6846	37.98	633	1.2291	0.7775	0.0775	0.535
1.6846	39.0	650	1.1773	0.7925	0.08	0.5425
1.6846	39.96	666	1.1908	0.79	0.08	0.5375
1.6846	40.98	683	1.2103	0.7875	0.0825	0.5375
1.6846	42.0	700	1.2058	0.795	0.085	0.545
1.6846	42.96	716	1.2105	0.7925	0.085	0.55
1.6846	43.98	733	1.2077	0.8025	0.0825	0.5425
1.6846	45.0	750	1.2358	0.79	0.0825	0.5375
1.6846	45.96	766	1.2305	0.7975	0.08	0.55
1.6846	46.98	783	1.2409	0.79	0.08	0.5525
1.6846	48.0	800	1.2810	0.785	0.08	0.55
1.6846	48.96	816	1.2593	0.7925	0.0825	0.5575
1.6846	49.98	833	1.2397	0.7975	0.0825	0.5575
1.6846	51.0	850	1.2456	0.795	0.0825	0.56
1.6846	51.96	866	1.2508	0.7975	0.0825	0.565
1.6846	52.98	883	1.2577	0.7975	0.085	0.5675
1.6846	54.0	900	1.2537	0.7975	0.085	0.56
1.6846	54.96	916	1.2552	0.7975	0.085	0.57
1.6846	55.98	933	1.2520	0.795	0.085	0.5675
1.6846	57.0	950	1.2478	0.795	0.085	0.57
1.6846	57.6	960	1.2478	0.795	0.085	0.57

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3