lmurray
/

layoutlm-funsd

@@ -17,14 +17,14 @@ should probably proofread and complete it, then remove this comment. -->
 This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.6953
-- Answer: {'precision': 0.7040261153427638, 'recall': 0.799752781211372, 'f1': 0.7488425925925926, 'number': 809}
-- Header: {'precision': 0.24516129032258063, 'recall': 0.31932773109243695, 'f1': 0.2773722627737226, 'number': 119}
-- Question: {'precision': 0.7894273127753304, 'recall': 0.8413145539906103, 'f1': 0.8145454545454545, 'number': 1065}
-- Overall Precision: 0.7157
-- Overall Recall: 0.7933
-- Overall F1: 0.7525
-- Overall Accuracy: 0.8063
 ## Model description
@@ -54,23 +54,23 @@ The following hyperparameters were used during training:
 ### Training results
-| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                        | Header                                                                                                        | Question                                                                                                     | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
-|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
-| 1.8542        | 1.0   | 10   | 1.6365          | {'precision': 0.01658374792703151, 'recall': 0.012360939431396786, 'f1': 0.014164305949008497, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.21144674085850557, 'recall': 0.12488262910798122, 'f1': 0.15702479338842976, 'number': 1065} | 0.1161            | 0.0718         | 0.0887     | 0.3298           |
-| 1.4883        | 2.0   | 20   | 1.3085          | {'precision': 0.17417417417417416, 'recall': 0.21508034610630408, 'f1': 0.19247787610619468, 'number': 809}   | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.37554585152838427, 'recall': 0.48450704225352115, 'f1': 0.4231242312423124, 'number': 1065}  | 0.2908            | 0.3462         | 0.3161     | 0.5523           |
-| 1.1521        | 3.0   | 30   | 0.9728          | {'precision': 0.452642073778664, 'recall': 0.5611866501854141, 'f1': 0.5011037527593818, 'number': 809}       | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.5948678071539658, 'recall': 0.7183098591549296, 'f1': 0.6507868991918333, 'number': 1065}    | 0.53              | 0.6116         | 0.5679     | 0.6953           |
-| 0.8808        | 4.0   | 40   | 0.7922          | {'precision': 0.582089552238806, 'recall': 0.723114956736712, 'f1': 0.6449834619625138, 'number': 809}        | {'precision': 0.06666666666666667, 'recall': 0.025210084033613446, 'f1': 0.036585365853658534, 'number': 119} | {'precision': 0.6649916247906198, 'recall': 0.7455399061032864, 'f1': 0.7029659141212926, 'number': 1065}    | 0.6159            | 0.6934         | 0.6523     | 0.7508           |
-| 0.6941        | 5.0   | 50   | 0.7105          | {'precision': 0.6461704422869471, 'recall': 0.7404202719406675, 'f1': 0.6900921658986174, 'number': 809}      | {'precision': 0.1875, 'recall': 0.15126050420168066, 'f1': 0.16744186046511625, 'number': 119}                | {'precision': 0.651778955336866, 'recall': 0.8084507042253521, 'f1': 0.721709974853311, 'number': 1065}      | 0.6305            | 0.7416         | 0.6816     | 0.7809           |
-| 0.5844        | 6.0   | 60   | 0.6780          | {'precision': 0.6598360655737705, 'recall': 0.796044499381953, 'f1': 0.7215686274509804, 'number': 809}       | {'precision': 0.2079207920792079, 'recall': 0.17647058823529413, 'f1': 0.19090909090909092, 'number': 119}    | {'precision': 0.7283633247643531, 'recall': 0.7981220657276995, 'f1': 0.7616487455197132, 'number': 1065}    | 0.6751            | 0.7602         | 0.7151     | 0.7926           |
-| 0.5021        | 7.0   | 70   | 0.6522          | {'precision': 0.6852248394004282, 'recall': 0.7911001236093943, 'f1': 0.7343660355708548, 'number': 809}      | {'precision': 0.20149253731343283, 'recall': 0.226890756302521, 'f1': 0.21343873517786563, 'number': 119}     | {'precision': 0.7521514629948365, 'recall': 0.8206572769953052, 'f1': 0.7849124382577458, 'number': 1065}    | 0.6910            | 0.7732         | 0.7298     | 0.8036           |
-| 0.443         | 8.0   | 80   | 0.6501          | {'precision': 0.6827731092436975, 'recall': 0.8034610630407911, 'f1': 0.7382169222032936, 'number': 809}      | {'precision': 0.2542372881355932, 'recall': 0.25210084033613445, 'f1': 0.25316455696202533, 'number': 119}    | {'precision': 0.7689003436426117, 'recall': 0.8403755868544601, 'f1': 0.8030506953790938, 'number': 1065}    | 0.7050            | 0.7903         | 0.7452     | 0.8060           |
-| 0.3917        | 9.0   | 90   | 0.6715          | {'precision': 0.6913319238900634, 'recall': 0.8084054388133498, 'f1': 0.7452991452991453, 'number': 809}      | {'precision': 0.25547445255474455, 'recall': 0.29411764705882354, 'f1': 0.2734375, 'number': 119}             | {'precision': 0.7811387900355872, 'recall': 0.8244131455399061, 'f1': 0.8021927820922796, 'number': 1065}    | 0.7100            | 0.7863         | 0.7462     | 0.8032           |
-| 0.3849        | 10.0  | 100  | 0.6725          | {'precision': 0.6908315565031983, 'recall': 0.8009888751545118, 'f1': 0.7418431597023468, 'number': 809}      | {'precision': 0.24444444444444444, 'recall': 0.2773109243697479, 'f1': 0.25984251968503935, 'number': 119}    | {'precision': 0.7829937998228521, 'recall': 0.8300469483568075, 'f1': 0.805834092980857, 'number': 1065}     | 0.7107            | 0.7852         | 0.7461     | 0.8064           |
-| 0.3232        | 11.0  | 110  | 0.6747          | {'precision': 0.6918976545842217, 'recall': 0.8022249690976514, 'f1': 0.7429879793932456, 'number': 809}      | {'precision': 0.25161290322580643, 'recall': 0.3277310924369748, 'f1': 0.2846715328467153, 'number': 119}     | {'precision': 0.7609797297297297, 'recall': 0.8460093896713615, 'f1': 0.8012449977767896, 'number': 1065}    | 0.6978            | 0.7973         | 0.7443     | 0.8001           |
-| 0.3028        | 12.0  | 120  | 0.6871          | {'precision': 0.700218818380744, 'recall': 0.7911001236093943, 'f1': 0.7428903076030179, 'number': 809}       | {'precision': 0.25, 'recall': 0.31092436974789917, 'f1': 0.27715355805243447, 'number': 119}                  | {'precision': 0.7985611510791367, 'recall': 0.8338028169014085, 'f1': 0.8158015617822691, 'number': 1065}    | 0.7199            | 0.7852         | 0.7511     | 0.8042           |
-| 0.284         | 13.0  | 130  | 0.6905          | {'precision': 0.697524219590958, 'recall': 0.8009888751545118, 'f1': 0.7456846950517838, 'number': 809}       | {'precision': 0.2602739726027397, 'recall': 0.31932773109243695, 'f1': 0.28679245283018867, 'number': 119}    | {'precision': 0.7929203539823009, 'recall': 0.8413145539906103, 'f1': 0.8164009111617311, 'number': 1065}    | 0.7175            | 0.7938         | 0.7537     | 0.8057           |
-| 0.2666        | 14.0  | 140  | 0.6958          | {'precision': 0.6949516648764769, 'recall': 0.799752781211372, 'f1': 0.7436781609195402, 'number': 809}       | {'precision': 0.2585034013605442, 'recall': 0.31932773109243695, 'f1': 0.2857142857142857, 'number': 119}     | {'precision': 0.7904085257548845, 'recall': 0.8356807511737089, 'f1': 0.8124144226380648, 'number': 1065}    | 0.7146            | 0.7903         | 0.7505     | 0.8040           |
-| 0.2705        | 15.0  | 150  | 0.6953          | {'precision': 0.7040261153427638, 'recall': 0.799752781211372, 'f1': 0.7488425925925926, 'number': 809}       | {'precision': 0.24516129032258063, 'recall': 0.31932773109243695, 'f1': 0.2773722627737226, 'number': 119}    | {'precision': 0.7894273127753304, 'recall': 0.8413145539906103, 'f1': 0.8145454545454545, 'number': 1065}    | 0.7157            | 0.7933         | 0.7525     | 0.8063           |
 ### Framework versions

 This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
 It achieves the following results on the evaluation set:
+- Loss: 0.7271
+- Answer: {'precision': 0.7209821428571429, 'recall': 0.7985166872682324, 'f1': 0.7577712609970675, 'number': 809}
+- Header: {'precision': 0.3308270676691729, 'recall': 0.3697478991596639, 'f1': 0.3492063492063492, 'number': 119}
+- Question: {'precision': 0.7732049036777583, 'recall': 0.8291079812206573, 'f1': 0.8001812415043046, 'number': 1065}
+- Overall Precision: 0.7246
+- Overall Recall: 0.7893
+- Overall F1: 0.7555
+- Overall Accuracy: 0.8054
 ## Model description
 ### Training results
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                      | Header                                                                                                       | Question                                                                                                     | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.8393        | 1.0   | 10   | 1.5994          | {'precision': 0.02424942263279446, 'recall': 0.02595797280593325, 'f1': 0.02507462686567164, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                  | {'precision': 0.22250639386189258, 'recall': 0.16338028169014085, 'f1': 0.18841364374661615, 'number': 1065} | 0.1183            | 0.0978         | 0.1071     | 0.3781           |
+| 1.4614        | 2.0   | 20   | 1.2520          | {'precision': 0.121765601217656, 'recall': 0.09888751545117429, 'f1': 0.10914051841746247, 'number': 809}   | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                  | {'precision': 0.47543713572023316, 'recall': 0.536150234741784, 'f1': 0.503971756398941, 'number': 1065}     | 0.3504            | 0.3266         | 0.3381     | 0.5844           |
+| 1.1144        | 3.0   | 30   | 0.9610          | {'precision': 0.5, 'recall': 0.5278121137206427, 'f1': 0.513529765484065, 'number': 809}                    | {'precision': 0.037037037037037035, 'recall': 0.008403361344537815, 'f1': 0.0136986301369863, 'number': 119} | {'precision': 0.6224758560140474, 'recall': 0.6657276995305165, 'f1': 0.6433756805807623, 'number': 1065}    | 0.5629            | 0.5705         | 0.5667     | 0.7246           |
+| 0.8509        | 4.0   | 40   | 0.7961          | {'precision': 0.5943396226415094, 'recall': 0.7008652657601978, 'f1': 0.6432217810550198, 'number': 809}    | {'precision': 0.3, 'recall': 0.12605042016806722, 'f1': 0.17751479289940827, 'number': 119}                  | {'precision': 0.6532188841201717, 'recall': 0.7145539906103286, 'f1': 0.6825112107623318, 'number': 1065}    | 0.6192            | 0.6739         | 0.6454     | 0.7543           |
+| 0.6724        | 5.0   | 50   | 0.7346          | {'precision': 0.6362683438155137, 'recall': 0.7503090234857849, 'f1': 0.6885989790130459, 'number': 809}    | {'precision': 0.3466666666666667, 'recall': 0.2184873949579832, 'f1': 0.26804123711340205, 'number': 119}    | {'precision': 0.6597444089456869, 'recall': 0.7755868544600939, 'f1': 0.7129909365558912, 'number': 1065}    | 0.6396            | 0.7321         | 0.6827     | 0.7797           |
+| 0.5743        | 6.0   | 60   | 0.7086          | {'precision': 0.6481481481481481, 'recall': 0.7787391841779975, 'f1': 0.7074677147669849, 'number': 809}    | {'precision': 0.3424657534246575, 'recall': 0.21008403361344538, 'f1': 0.2604166666666667, 'number': 119}    | {'precision': 0.7128116938950989, 'recall': 0.7784037558685446, 'f1': 0.7441651705565528, 'number': 1065}    | 0.6721            | 0.7446         | 0.7065     | 0.7845           |
+| 0.4963        | 7.0   | 70   | 0.6881          | {'precision': 0.6866952789699571, 'recall': 0.7911001236093943, 'f1': 0.7352096496266514, 'number': 809}    | {'precision': 0.30392156862745096, 'recall': 0.2605042016806723, 'f1': 0.28054298642533937, 'number': 119}   | {'precision': 0.7263249348392702, 'recall': 0.7849765258215963, 'f1': 0.7545126353790614, 'number': 1065}    | 0.6897            | 0.7561         | 0.7214     | 0.7926           |
+| 0.4392        | 8.0   | 80   | 0.7116          | {'precision': 0.6779487179487179, 'recall': 0.8170580964153276, 'f1': 0.741031390134529, 'number': 809}     | {'precision': 0.28431372549019607, 'recall': 0.24369747899159663, 'f1': 0.26244343891402716, 'number': 119}  | {'precision': 0.7322175732217573, 'recall': 0.8215962441314554, 'f1': 0.7743362831858407, 'number': 1065}    | 0.6888            | 0.7852         | 0.7339     | 0.7886           |
+| 0.3945        | 9.0   | 90   | 0.7000          | {'precision': 0.7060737527114967, 'recall': 0.8046971569839307, 'f1': 0.7521663778162911, 'number': 809}    | {'precision': 0.2920353982300885, 'recall': 0.2773109243697479, 'f1': 0.28448275862068967, 'number': 119}    | {'precision': 0.7502183406113537, 'recall': 0.8065727699530516, 'f1': 0.7773755656108599, 'number': 1065}    | 0.7078            | 0.7742         | 0.7395     | 0.7989           |
+| 0.3825        | 10.0  | 100  | 0.7006          | {'precision': 0.717607973421927, 'recall': 0.8009888751545118, 'f1': 0.7570093457943926, 'number': 809}     | {'precision': 0.29464285714285715, 'recall': 0.2773109243697479, 'f1': 0.28571428571428575, 'number': 119}   | {'precision': 0.7642418930762489, 'recall': 0.8187793427230047, 'f1': 0.7905711695376247, 'number': 1065}    | 0.7203            | 0.7792         | 0.7486     | 0.8053           |
+| 0.327         | 11.0  | 110  | 0.7180          | {'precision': 0.6969376979936642, 'recall': 0.8158220024721878, 'f1': 0.7517084282460138, 'number': 809}    | {'precision': 0.2975206611570248, 'recall': 0.3025210084033613, 'f1': 0.3, 'number': 119}                    | {'precision': 0.7572898799313894, 'recall': 0.8291079812206573, 'f1': 0.7915732855221873, 'number': 1065}    | 0.7068            | 0.7923         | 0.7471     | 0.7969           |
+| 0.3136        | 12.0  | 120  | 0.7147          | {'precision': 0.7283950617283951, 'recall': 0.8022249690976514, 'f1': 0.7635294117647059, 'number': 809}    | {'precision': 0.3305084745762712, 'recall': 0.3277310924369748, 'f1': 0.32911392405063294, 'number': 119}    | {'precision': 0.7831111111111111, 'recall': 0.8272300469483568, 'f1': 0.8045662100456622, 'number': 1065}    | 0.7352            | 0.7873         | 0.7604     | 0.8059           |
+| 0.2943        | 13.0  | 130  | 0.7297          | {'precision': 0.7136659436008677, 'recall': 0.8133498145859085, 'f1': 0.7602541883304449, 'number': 809}    | {'precision': 0.34210526315789475, 'recall': 0.3277310924369748, 'f1': 0.33476394849785407, 'number': 119}   | {'precision': 0.7785588752196837, 'recall': 0.831924882629108, 'f1': 0.8043576940535634, 'number': 1065}     | 0.7282            | 0.7943         | 0.7598     | 0.8001           |
+| 0.2727        | 14.0  | 140  | 0.7284          | {'precision': 0.7275784753363229, 'recall': 0.8022249690976514, 'f1': 0.7630805408583187, 'number': 809}    | {'precision': 0.3333333333333333, 'recall': 0.3697478991596639, 'f1': 0.350597609561753, 'number': 119}      | {'precision': 0.7732049036777583, 'recall': 0.8291079812206573, 'f1': 0.8001812415043046, 'number': 1065}    | 0.7276            | 0.7908         | 0.7579     | 0.8046           |
+| 0.2753        | 15.0  | 150  | 0.7271          | {'precision': 0.7209821428571429, 'recall': 0.7985166872682324, 'f1': 0.7577712609970675, 'number': 809}    | {'precision': 0.3308270676691729, 'recall': 0.3697478991596639, 'f1': 0.3492063492063492, 'number': 119}     | {'precision': 0.7732049036777583, 'recall': 0.8291079812206573, 'f1': 0.8001812415043046, 'number': 1065}    | 0.7246            | 0.7893         | 0.7555     | 0.8054           |
 ### Framework versions

logs/events.out.tfevents.1719274259.ae49b29c4439.9416.0 CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:abe4cdd20128f488715bc627d0f6245aaa5e2296fc7173d892c015aa540c1204
-size 12780

 version https://git-lfs.github.com/spec/v1
+oid sha256:a6235f33095f3479b70bf7004214763b5581252f40a932b2788950ab99dc2da7
+size 15981

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3ea009ad7e31229b768c477e66371bdafcd909ee27db082a4207f829eae7192a
 size 450558212

 version https://git-lfs.github.com/spec/v1
+oid sha256:0b6ea22986c663625b657a9c3a1958089e7df765726ebb6def2d163687a408f5
 size 450558212