EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-12-04_txt_vis_concat_enc_8_gate

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0199
• Accuracy: 0.7575
• Exit 0 Accuracy: 0.06
• Exit 1 Accuracy: 0.12

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 192
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	4	2.7540	0.09	0.0425	0.0625
No log	1.96	8	2.7063	0.1525	0.045	0.0625
No log	2.96	12	2.6279	0.1775	0.05	0.0625
No log	3.96	16	2.5448	0.2025	0.0525	0.0625
No log	4.96	20	2.4829	0.225	0.05	0.0625
No log	5.96	24	2.3776	0.285	0.0525	0.0625
No log	6.96	28	2.2785	0.33	0.055	0.0625
No log	7.96	32	2.1809	0.3575	0.055	0.0625
No log	8.96	36	2.0830	0.3875	0.055	0.085
No log	9.96	40	1.9854	0.4275	0.055	0.07
No log	10.96	44	1.8839	0.485	0.055	0.07
No log	11.96	48	1.7375	0.545	0.055	0.0775
No log	12.96	52	1.6524	0.5725	0.0575	0.095
No log	13.96	56	1.5703	0.565	0.0575	0.1175
No log	14.96	60	1.4606	0.6175	0.0575	0.1175
No log	15.96	64	1.3988	0.6525	0.055	0.1175
No log	16.96	68	1.3036	0.645	0.055	0.1175
No log	17.96	72	1.2528	0.6875	0.0575	0.12
No log	18.96	76	1.2113	0.69	0.0575	0.12
No log	19.96	80	1.1515	0.6975	0.0575	0.1175
No log	20.96	84	1.1117	0.71	0.0575	0.1175
No log	21.96	88	1.0811	0.7075	0.0575	0.1175
No log	22.96	92	1.0649	0.7075	0.0575	0.1175
No log	23.96	96	1.0342	0.7275	0.0575	0.1175
No log	24.96	100	0.9994	0.7275	0.0575	0.1175
No log	25.96	104	1.0332	0.7275	0.0575	0.1175
No log	26.96	108	0.9696	0.7325	0.06	0.1175
No log	27.96	112	0.9957	0.7325	0.06	0.1175
No log	28.96	116	0.9794	0.7325	0.06	0.1175
No log	29.96	120	0.9602	0.735	0.06	0.1175
No log	30.96	124	0.9900	0.75	0.06	0.12
No log	31.96	128	0.9649	0.74	0.06	0.1175
No log	32.96	132	0.9935	0.7325	0.06	0.12
No log	33.96	136	0.9637	0.7475	0.06	0.12
No log	34.96	140	0.9993	0.7325	0.06	0.12
No log	35.96	144	0.9841	0.7375	0.06	0.1225
No log	36.96	148	0.9719	0.7475	0.06	0.1225
No log	37.96	152	0.9901	0.745	0.06	0.125
No log	38.96	156	0.9692	0.7475	0.06	0.115
No log	39.96	160	0.9850	0.7425	0.06	0.1025
No log	40.96	164	0.9876	0.75	0.06	0.1
No log	41.96	168	0.9867	0.7525	0.06	0.095
No log	42.96	172	1.0042	0.7475	0.06	0.085
No log	43.96	176	1.0099	0.735	0.06	0.1025
No log	44.96	180	0.9988	0.755	0.06	0.1025
No log	45.96	184	1.0033	0.7475	0.06	0.1025
No log	46.96	188	1.0105	0.7475	0.06	0.105
No log	47.96	192	1.0103	0.75	0.06	0.1025
No log	48.96	196	1.0097	0.7575	0.06	0.11
No log	49.96	200	0.9975	0.755	0.06	0.11
No log	50.96	204	1.0095	0.76	0.06	0.115
No log	51.96	208	1.0203	0.7575	0.06	0.1275
No log	52.96	212	1.0304	0.745	0.06	0.1225
No log	53.96	216	1.0286	0.75	0.06	0.125
No log	54.96	220	1.0193	0.7575	0.06	0.1275
No log	55.96	224	1.0138	0.7575	0.06	0.13
No log	56.96	228	1.0168	0.76	0.06	0.1225
No log	57.96	232	1.0196	0.76	0.06	0.12
No log	58.96	236	1.0203	0.7575	0.06	0.12
No log	59.96	240	1.0199	0.7575	0.06	0.12

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1.post200
• Datasets 2.9.0
• Tokenizers 0.13.2