layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7376
• Answer: {'precision': 0.6796116504854369, 'recall': 0.7787391841779975, 'f1': 0.7258064516129031, 'number': 809}
• Header: {'precision': 0.3076923076923077, 'recall': 0.33613445378151263, 'f1': 0.321285140562249, 'number': 119}
• Question: {'precision': 0.7692307692307693, 'recall': 0.8356807511737089, 'f1': 0.8010801080108011, 'number': 1065}
• Overall Precision: 0.7046
• Overall Recall: 0.7827
• Overall F1: 0.7416
• Overall Accuracy: 0.7934

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8071	1.0	10	1.6362	{'precision': 0.01038961038961039, 'recall': 0.004944375772558714, 'f1': 0.006700167504187604, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.336, 'recall': 0.11830985915492957, 'f1': 0.17500000000000002, 'number': 1065}	0.1711	0.0652	0.0944	0.3269
1.5124	2.0	20	1.3120	{'precision': 0.12598425196850394, 'recall': 0.11866501854140915, 'f1': 0.12221514958625079, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4277456647398844, 'recall': 0.4863849765258216, 'f1': 0.4551845342706503, 'number': 1065}	0.3112	0.3081	0.3096	0.5731
1.1643	3.0	30	1.0389	{'precision': 0.4329411764705882, 'recall': 0.45488257107540175, 'f1': 0.44364074743821574, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5088122605363985, 'recall': 0.6234741784037559, 'f1': 0.560337552742616, 'number': 1065}	0.4727	0.5178	0.4943	0.6785
0.8925	4.0	40	0.8687	{'precision': 0.5737704918032787, 'recall': 0.6489493201483313, 'f1': 0.6090487238979118, 'number': 809}	{'precision': 0.2222222222222222, 'recall': 0.08403361344537816, 'f1': 0.1219512195121951, 'number': 119}	{'precision': 0.664568345323741, 'recall': 0.6938967136150235, 'f1': 0.6789159393661002, 'number': 1065}	0.6149	0.6392	0.6268	0.7348
0.7169	5.0	50	0.7748	{'precision': 0.6039707419017764, 'recall': 0.7144622991347342, 'f1': 0.6545866364665911, 'number': 809}	{'precision': 0.25274725274725274, 'recall': 0.19327731092436976, 'f1': 0.21904761904761905, 'number': 119}	{'precision': 0.6873935264054515, 'recall': 0.7577464788732394, 'f1': 0.7208575256811076, 'number': 1065}	0.6337	0.7065	0.6681	0.7647
0.5923	6.0	60	0.7214	{'precision': 0.6291322314049587, 'recall': 0.7527812113720643, 'f1': 0.6854248733821047, 'number': 809}	{'precision': 0.323943661971831, 'recall': 0.19327731092436976, 'f1': 0.24210526315789474, 'number': 119}	{'precision': 0.6884735202492211, 'recall': 0.8300469483568075, 'f1': 0.752660706683695, 'number': 1065}	0.6526	0.7607	0.7025	0.7795
0.5208	7.0	70	0.7338	{'precision': 0.6555793991416309, 'recall': 0.7552533992583437, 'f1': 0.7018954623779436, 'number': 809}	{'precision': 0.27450980392156865, 'recall': 0.23529411764705882, 'f1': 0.2533936651583711, 'number': 119}	{'precision': 0.7224523612261806, 'recall': 0.8187793427230047, 'f1': 0.7676056338028169, 'number': 1065}	0.6743	0.7582	0.7137	0.7853
0.4668	8.0	80	0.6981	{'precision': 0.6534446764091858, 'recall': 0.7737948084054388, 'f1': 0.708545557441992, 'number': 809}	{'precision': 0.27884615384615385, 'recall': 0.24369747899159663, 'f1': 0.2600896860986547, 'number': 119}	{'precision': 0.7394190871369295, 'recall': 0.8366197183098592, 'f1': 0.785022026431718, 'number': 1065}	0.6820	0.7757	0.7258	0.7917
0.413	9.0	90	0.7140	{'precision': 0.6777408637873754, 'recall': 0.7564894932014833, 'f1': 0.7149532710280373, 'number': 809}	{'precision': 0.2755905511811024, 'recall': 0.29411764705882354, 'f1': 0.2845528455284553, 'number': 119}	{'precision': 0.7495784148397976, 'recall': 0.8347417840375587, 'f1': 0.7898711683696135, 'number': 1065}	0.6931	0.7707	0.7299	0.7910
0.372	10.0	100	0.7031	{'precision': 0.6843853820598007, 'recall': 0.7639060568603214, 'f1': 0.72196261682243, 'number': 809}	{'precision': 0.31092436974789917, 'recall': 0.31092436974789917, 'f1': 0.31092436974789917, 'number': 119}	{'precision': 0.749793559042114, 'recall': 0.8525821596244132, 'f1': 0.7978910369068541, 'number': 1065}	0.7000	0.7842	0.7397	0.7998
0.3317	11.0	110	0.7231	{'precision': 0.677765843179377, 'recall': 0.7799752781211372, 'f1': 0.725287356321839, 'number': 809}	{'precision': 0.3162393162393162, 'recall': 0.31092436974789917, 'f1': 0.3135593220338983, 'number': 119}	{'precision': 0.7748917748917749, 'recall': 0.8403755868544601, 'f1': 0.8063063063063063, 'number': 1065}	0.7095	0.7842	0.7450	0.7931
0.3156	12.0	120	0.7385	{'precision': 0.6800433839479393, 'recall': 0.7750309023485785, 'f1': 0.7244367417677644, 'number': 809}	{'precision': 0.3253968253968254, 'recall': 0.3445378151260504, 'f1': 0.33469387755102037, 'number': 119}	{'precision': 0.7756521739130435, 'recall': 0.8375586854460094, 'f1': 0.8054176072234764, 'number': 1065}	0.7097	0.7827	0.7445	0.7924
0.2945	13.0	130	0.7311	{'precision': 0.6856516976998904, 'recall': 0.7737948084054388, 'f1': 0.727061556329849, 'number': 809}	{'precision': 0.3203125, 'recall': 0.3445378151260504, 'f1': 0.33198380566801616, 'number': 119}	{'precision': 0.7658662092624356, 'recall': 0.8384976525821596, 'f1': 0.8005378753922008, 'number': 1065}	0.7068	0.7827	0.7429	0.7937
0.2867	14.0	140	0.7304	{'precision': 0.6826086956521739, 'recall': 0.7762669962917181, 'f1': 0.7264314632735686, 'number': 809}	{'precision': 0.31496062992125984, 'recall': 0.33613445378151263, 'f1': 0.3252032520325203, 'number': 119}	{'precision': 0.7632933104631218, 'recall': 0.8356807511737089, 'f1': 0.7978484984311968, 'number': 1065}	0.7040	0.7817	0.7408	0.7934
0.2865	15.0	150	0.7376	{'precision': 0.6796116504854369, 'recall': 0.7787391841779975, 'f1': 0.7258064516129031, 'number': 809}	{'precision': 0.3076923076923077, 'recall': 0.33613445378151263, 'f1': 0.321285140562249, 'number': 119}	{'precision': 0.7692307692307693, 'recall': 0.8356807511737089, 'f1': 0.8010801080108011, 'number': 1065}	0.7046	0.7827	0.7416	0.7934

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3