layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6845
• Answer: {'precision': 0.6932907348242812, 'recall': 0.8046971569839307, 'f1': 0.7448512585812357, 'number': 809}
• Header: {'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119}
• Question: {'precision': 0.7827225130890052, 'recall': 0.8422535211267606, 'f1': 0.8113975576662144, 'number': 1065}
• Overall Precision: 0.7199
• Overall Recall: 0.7958
• Overall F1: 0.7560
• Overall Accuracy: 0.8087

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7948	1.0	10	1.5982	{'precision': 0.019115890083632018, 'recall': 0.019777503090234856, 'f1': 0.01944106925880923, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.1559202813599062, 'recall': 0.12488262910798122, 'f1': 0.1386861313868613, 'number': 1065}	0.0882	0.0748	0.0809	0.3666
1.4548	2.0	20	1.2137	{'precision': 0.18571428571428572, 'recall': 0.19283065512978986, 'f1': 0.18920557913887204, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5027844073190135, 'recall': 0.5934272300469483, 'f1': 0.5443583118001722, 'number': 1065}	0.3758	0.3954	0.3853	0.6060
1.0759	3.0	30	0.9074	{'precision': 0.45133689839572194, 'recall': 0.5216316440049443, 'f1': 0.48394495412844035, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6371453138435081, 'recall': 0.6957746478873239, 'f1': 0.6651705565529622, 'number': 1065}	0.5491	0.5835	0.5658	0.7138
0.818	4.0	40	0.7636	{'precision': 0.601010101010101, 'recall': 0.7354758961681088, 'f1': 0.6614785992217899, 'number': 809}	{'precision': 0.22, 'recall': 0.09243697478991597, 'f1': 0.13017751479289943, 'number': 119}	{'precision': 0.6860670194003528, 'recall': 0.7305164319248826, 'f1': 0.707594361073215, 'number': 1065}	0.6366	0.6944	0.6643	0.7580
0.6744	5.0	50	0.6948	{'precision': 0.6172106824925816, 'recall': 0.7713226205191595, 'f1': 0.6857142857142857, 'number': 809}	{'precision': 0.2608695652173913, 'recall': 0.15126050420168066, 'f1': 0.19148936170212766, 'number': 119}	{'precision': 0.7063758389261745, 'recall': 0.7906103286384977, 'f1': 0.7461231723526807, 'number': 1065}	0.6532	0.7446	0.6959	0.7803
0.5678	6.0	60	0.6772	{'precision': 0.6684100418410042, 'recall': 0.7898640296662547, 'f1': 0.7240793201133144, 'number': 809}	{'precision': 0.32857142857142857, 'recall': 0.19327731092436976, 'f1': 0.2433862433862434, 'number': 119}	{'precision': 0.7155309033280507, 'recall': 0.847887323943662, 'f1': 0.7761065749892565, 'number': 1065}	0.6840	0.7852	0.7311	0.7902
0.4886	7.0	70	0.6596	{'precision': 0.6836518046709129, 'recall': 0.796044499381953, 'f1': 0.7355796687607081, 'number': 809}	{'precision': 0.30952380952380953, 'recall': 0.2184873949579832, 'f1': 0.2561576354679803, 'number': 119}	{'precision': 0.75, 'recall': 0.8422535211267606, 'f1': 0.793454223794781, 'number': 1065}	0.7052	0.7863	0.7435	0.7931
0.4432	8.0	80	0.6525	{'precision': 0.6792849631966351, 'recall': 0.7985166872682324, 'f1': 0.734090909090909, 'number': 809}	{'precision': 0.2736842105263158, 'recall': 0.2184873949579832, 'f1': 0.2429906542056075, 'number': 119}	{'precision': 0.7472984206151289, 'recall': 0.844131455399061, 'f1': 0.7927689594356261, 'number': 1065}	0.6985	0.7883	0.7407	0.7965
0.3961	9.0	90	0.6515	{'precision': 0.6940540540540541, 'recall': 0.7935723114956736, 'f1': 0.740484429065744, 'number': 809}	{'precision': 0.2803738317757009, 'recall': 0.25210084033613445, 'f1': 0.2654867256637167, 'number': 119}	{'precision': 0.7613344739093242, 'recall': 0.8356807511737089, 'f1': 0.7967770814682185, 'number': 1065}	0.7097	0.7837	0.7449	0.8019
0.3531	10.0	100	0.6628	{'precision': 0.697452229299363, 'recall': 0.8121137206427689, 'f1': 0.750428326670474, 'number': 809}	{'precision': 0.2962962962962963, 'recall': 0.2689075630252101, 'f1': 0.28193832599118945, 'number': 119}	{'precision': 0.7577276524644946, 'recall': 0.8516431924882629, 'f1': 0.801945181255526, 'number': 1065}	0.7103	0.8008	0.7528	0.8034
0.3201	11.0	110	0.6678	{'precision': 0.6915005246589717, 'recall': 0.8145859085290482, 'f1': 0.7480136208853576, 'number': 809}	{'precision': 0.2909090909090909, 'recall': 0.2689075630252101, 'f1': 0.2794759825327511, 'number': 119}	{'precision': 0.7679794520547946, 'recall': 0.8422535211267606, 'f1': 0.8034034930586654, 'number': 1065}	0.7118	0.7968	0.7519	0.8071
0.3055	12.0	120	0.6760	{'precision': 0.6869747899159664, 'recall': 0.8084054388133498, 'f1': 0.7427597955706984, 'number': 809}	{'precision': 0.296, 'recall': 0.31092436974789917, 'f1': 0.30327868852459017, 'number': 119}	{'precision': 0.7839506172839507, 'recall': 0.8347417840375587, 'f1': 0.8085493406093679, 'number': 1065}	0.7146	0.7928	0.7517	0.8047
0.29	13.0	130	0.6844	{'precision': 0.7013963480128894, 'recall': 0.8071693448702101, 'f1': 0.7505747126436783, 'number': 809}	{'precision': 0.28346456692913385, 'recall': 0.3025210084033613, 'f1': 0.2926829268292683, 'number': 119}	{'precision': 0.7771084337349398, 'recall': 0.847887323943662, 'f1': 0.8109564436461607, 'number': 1065}	0.7171	0.7988	0.7558	0.8041
0.2797	14.0	140	0.6841	{'precision': 0.6956055734190782, 'recall': 0.8022249690976514, 'f1': 0.7451205510907002, 'number': 809}	{'precision': 0.3064516129032258, 'recall': 0.31932773109243695, 'f1': 0.31275720164609055, 'number': 119}	{'precision': 0.7750865051903114, 'recall': 0.8413145539906103, 'f1': 0.8068437640702386, 'number': 1065}	0.7153	0.7943	0.7527	0.8070
0.2733	15.0	150	0.6845	{'precision': 0.6932907348242812, 'recall': 0.8046971569839307, 'f1': 0.7448512585812357, 'number': 809}	{'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119}	{'precision': 0.7827225130890052, 'recall': 0.8422535211267606, 'f1': 0.8113975576662144, 'number': 1065}	0.7199	0.7958	0.7560	0.8087

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 2.10.0
• Tokenizers 0.13.2