--- tags: - generated_from_trainer datasets: - funsd model-index: - name: layoutlm-funsd results: [] --- # layoutlm-funsd This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. It achieves the following results on the evaluation set: - Loss: 0.8261 - Answer: {'precision': 0.5727482678983834, 'recall': 0.6131025957972805, 'f1': 0.5922388059701492, 'number': 809} - Header: {'precision': 0.09302325581395349, 'recall': 0.03361344537815126, 'f1': 0.04938271604938272, 'number': 119} - Question: {'precision': 0.6384228187919463, 'recall': 0.7145539906103286, 'f1': 0.6743464776251661, 'number': 1065} - Overall Precision: 0.6002 - Overall Recall: 0.6327 - Overall F1: 0.6160 - Overall Accuracy: 0.7523 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 64 - eval_batch_size: 32 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 15 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| | 1.9264 | 1.0 | 3 | 1.7763 | {'precision': 0.011029411764705883, 'recall': 0.022249690976514216, 'f1': 0.01474805407619828, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.09483960948396095, 'recall': 0.12769953051643193, 'f1': 0.108843537414966, 'number': 1065} | 0.0483 | 0.0773 | 0.0595 | 0.3277 | | 1.7361 | 2.0 | 6 | 1.6376 | {'precision': 0.0064754856614246065, 'recall': 0.00865265760197775, 'f1': 0.007407407407407408, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.17735470941883769, 'recall': 0.16619718309859155, 'f1': 0.17159476490547748, 'number': 1065} | 0.0885 | 0.0923 | 0.0904 | 0.3852 | | 1.6212 | 3.0 | 9 | 1.5225 | {'precision': 0.02002002002002002, 'recall': 0.024721878862793572, 'f1': 0.022123893805309734, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.27049180327868855, 'recall': 0.27887323943661974, 'f1': 0.27461858529819694, 'number': 1065} | 0.1512 | 0.1591 | 0.1550 | 0.4422 | | 1.5178 | 4.0 | 12 | 1.4133 | {'precision': 0.05408388520971302, 'recall': 0.06056860321384425, 'f1': 0.05714285714285715, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.313953488372093, 'recall': 0.38028169014084506, 'f1': 0.34394904458598724, 'number': 1065} | 0.2067 | 0.2278 | 0.2168 | 0.5062 | | 1.3853 | 5.0 | 15 | 1.3086 | {'precision': 0.08221024258760108, 'recall': 0.0754017305315204, 'f1': 0.07865892972275951, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3780202650038971, 'recall': 0.45539906103286387, 'f1': 0.4131175468483816, 'number': 1065} | 0.2696 | 0.2740 | 0.2718 | 0.5453 | | 1.2546 | 6.0 | 18 | 1.2110 | {'precision': 0.1463768115942029, 'recall': 0.12484548825710753, 'f1': 0.13475650433622416, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4551282051282051, 'recall': 0.5333333333333333, 'f1': 0.49113705144833547, 'number': 1065} | 0.3452 | 0.3357 | 0.3404 | 0.5822 | | 1.1842 | 7.0 | 21 | 1.1217 | {'precision': 0.2563739376770538, 'recall': 0.22373300370828184, 'f1': 0.23894389438943894, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4908512330946698, 'recall': 0.5793427230046948, 'f1': 0.5314384151593453, 'number': 1065} | 0.4055 | 0.4004 | 0.4029 | 0.6223 | | 1.0564 | 8.0 | 24 | 1.0490 | {'precision': 0.364461738002594, 'recall': 0.3473423980222497, 'f1': 0.3556962025316456, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.509976057462091, 'recall': 0.6, 'f1': 0.551337359792925, 'number': 1065} | 0.4523 | 0.4616 | 0.4569 | 0.6679 | | 0.9865 | 9.0 | 27 | 0.9863 | {'precision': 0.4305555555555556, 'recall': 0.4215080346106304, 'f1': 0.42598376014990635, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5528726061615321, 'recall': 0.6234741784037559, 'f1': 0.5860547219770521, 'number': 1065} | 0.4978 | 0.5043 | 0.5010 | 0.6986 | | 0.9281 | 10.0 | 30 | 0.9357 | {'precision': 0.49454545454545457, 'recall': 0.5043263288009888, 'f1': 0.49938800489596086, 'number': 809} | {'precision': 0.034482758620689655, 'recall': 0.008403361344537815, 'f1': 0.013513513513513513, 'number': 119} | {'precision': 0.5873287671232876, 'recall': 0.644131455399061, 'f1': 0.6144200626959248, 'number': 1065} | 0.5415 | 0.5494 | 0.5455 | 0.7197 | | 0.8646 | 11.0 | 33 | 0.8968 | {'precision': 0.5333333333333333, 'recall': 0.5438813349814586, 'f1': 0.5385556915544676, 'number': 809} | {'precision': 0.0625, 'recall': 0.01680672268907563, 'f1': 0.026490066225165563, 'number': 119} | {'precision': 0.6031746031746031, 'recall': 0.6779342723004694, 'f1': 0.6383731211317418, 'number': 1065} | 0.5667 | 0.5840 | 0.5752 | 0.7344 | | 0.828 | 12.0 | 36 | 0.8653 | {'precision': 0.5617577197149644, 'recall': 0.584672435105068, 'f1': 0.5729860690490611, 'number': 809} | {'precision': 0.07692307692307693, 'recall': 0.025210084033613446, 'f1': 0.0379746835443038, 'number': 119} | {'precision': 0.6204013377926422, 'recall': 0.6967136150234742, 'f1': 0.6563467492260062, 'number': 1065} | 0.5864 | 0.6111 | 0.5985 | 0.7442 | | 0.7803 | 13.0 | 39 | 0.8442 | {'precision': 0.5667828106852497, 'recall': 0.6032138442521632, 'f1': 0.5844311377245508, 'number': 809} | {'precision': 0.07142857142857142, 'recall': 0.025210084033613446, 'f1': 0.037267080745341616, 'number': 119} | {'precision': 0.6343906510851419, 'recall': 0.7136150234741784, 'f1': 0.6716747680070703, 'number': 1065} | 0.5954 | 0.6277 | 0.6111 | 0.7504 | | 0.771 | 14.0 | 42 | 0.8312 | {'precision': 0.5679723502304147, 'recall': 0.6093943139678616, 'f1': 0.5879546809779368, 'number': 809} | {'precision': 0.09302325581395349, 'recall': 0.03361344537815126, 'f1': 0.04938271604938272, 'number': 119} | {'precision': 0.6376569037656904, 'recall': 0.7154929577464789, 'f1': 0.6743362831858407, 'number': 1065} | 0.5978 | 0.6317 | 0.6143 | 0.7516 | | 0.7843 | 15.0 | 45 | 0.8261 | {'precision': 0.5727482678983834, 'recall': 0.6131025957972805, 'f1': 0.5922388059701492, 'number': 809} | {'precision': 0.09302325581395349, 'recall': 0.03361344537815126, 'f1': 0.04938271604938272, 'number': 119} | {'precision': 0.6384228187919463, 'recall': 0.7145539906103286, 'f1': 0.6743464776251661, 'number': 1065} | 0.6002 | 0.6327 | 0.6160 | 0.7523 | ### Framework versions - Transformers 4.22.0.dev0 - Pytorch 1.12.1+cu116 - Datasets 2.4.0 - Tokenizers 0.12.1