EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-08-12_text_visual_concat_only

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1314
• Accuracy: 0.7925
• Exit 0 Accuracy: 0.075

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy
No log	0.96	16	2.6835	0.1175	0.0775
No log	1.98	33	2.4924	0.2375	0.0675
No log	3.0	50	2.2916	0.34	0.07
No log	3.96	66	2.0277	0.475	0.07
No log	4.98	83	1.7026	0.5775	0.07
No log	6.0	100	1.4354	0.6625	0.075
No log	6.96	116	1.2357	0.6975	0.07
No log	7.98	133	1.0754	0.745	0.0725
No log	9.0	150	0.9760	0.77	0.07
No log	9.96	166	0.8967	0.765	0.07
No log	10.98	183	0.8561	0.7875	0.07
No log	12.0	200	0.8837	0.7575	0.07
No log	12.96	216	0.8885	0.7725	0.07
No log	13.98	233	0.7819	0.7725	0.0675
No log	15.0	250	0.9067	0.7725	0.07
No log	15.96	266	0.9086	0.775	0.075
No log	16.98	283	0.8444	0.795	0.07
No log	18.0	300	0.9359	0.7775	0.0725
No log	18.96	316	0.9696	0.7825	0.07
No log	19.98	333	0.9254	0.7825	0.075
No log	21.0	350	0.9879	0.7775	0.0675
No log	21.96	366	0.9894	0.79	0.075
No log	22.98	383	1.0087	0.785	0.07
No log	24.0	400	1.0101	0.785	0.07
No log	24.96	416	1.0188	0.7875	0.075
No log	25.98	433	1.0266	0.79	0.075
No log	27.0	450	1.0357	0.79	0.07
No log	27.96	466	1.0505	0.7875	0.0675
No log	28.98	483	1.0524	0.7825	0.07
1.6927	30.0	500	1.0656	0.785	0.075
1.6927	30.96	516	1.0642	0.785	0.0725
1.6927	31.98	533	1.0740	0.785	0.0725
1.6927	33.0	550	1.0830	0.785	0.075
1.6927	33.96	566	1.0860	0.785	0.0675
1.6927	34.98	583	1.0939	0.7875	0.0675
1.6927	36.0	600	1.0969	0.79	0.0675
1.6927	36.96	616	1.0966	0.7875	0.07
1.6927	37.98	633	1.1024	0.79	0.07
1.6927	39.0	650	1.1045	0.7875	0.075
1.6927	39.96	666	1.1050	0.7875	0.0725
1.6927	40.98	683	1.1085	0.7875	0.075
1.6927	42.0	700	1.1171	0.7875	0.075
1.6927	42.96	716	1.1193	0.7875	0.075
1.6927	43.98	733	1.1188	0.79	0.075
1.6927	45.0	750	1.1220	0.79	0.0725
1.6927	45.96	766	1.1270	0.79	0.0725
1.6927	46.98	783	1.1268	0.7875	0.075
1.6927	48.0	800	1.1264	0.79	0.075
1.6927	48.96	816	1.1267	0.795	0.075
1.6927	49.98	833	1.1273	0.79	0.0725
1.6927	51.0	850	1.1268	0.79	0.07
1.6927	51.96	866	1.1283	0.795	0.07
1.6927	52.98	883	1.1293	0.795	0.075
1.6927	54.0	900	1.1306	0.795	0.075
1.6927	54.96	916	1.1306	0.795	0.075
1.6927	55.98	933	1.1310	0.795	0.075
1.6927	57.0	950	1.1314	0.7925	0.075
1.6927	57.6	960	1.1314	0.7925	0.075

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3