End of training

README.md

@@ -0,0 +1,81 @@
+---
+license: mit
+base_model: microsoft/layoutlm-base-uncased
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.0890
+- Answer: {'precision': 0.38420107719928187, 'recall': 0.5290482076637825, 'f1': 0.4451378055122205, 'number': 809}
+- Header: {'precision': 0.28888888888888886, 'recall': 0.2184873949579832, 'f1': 0.24880382775119617, 'number': 119}
+- Question: {'precision': 0.48959136468774095, 'recall': 0.596244131455399, 'f1': 0.5376799322607958, 'number': 1065}
+- Overall Precision: 0.4354
+- Overall Recall: 0.5464
+- Overall F1: 0.4846
+- Overall Accuracy: 0.6258
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                       | Header                                                                                                        | Question                                                                                                    | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.7643        | 1.0   | 10   | 1.5177          | {'precision': 0.052202283849918436, 'recall': 0.07911001236093942, 'f1': 0.06289926289926291, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.2581360946745562, 'recall': 0.3276995305164319, 'f1': 0.2887877534133223, 'number': 1065}   | 0.1602            | 0.2072         | 0.1807     | 0.3823           |
+| 1.4448        | 2.0   | 20   | 1.3359          | {'precision': 0.18779342723004694, 'recall': 0.39555006180469715, 'f1': 0.2546756864305611, 'number': 809}   | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.2733245729303548, 'recall': 0.39061032863849765, 'f1': 0.321608040201005, 'number': 1065}   | 0.2273            | 0.3693         | 0.2814     | 0.4206           |
+| 1.2967        | 3.0   | 30   | 1.2160          | {'precision': 0.2261437908496732, 'recall': 0.4276885043263288, 'f1': 0.29585292860196666, 'number': 809}    | {'precision': 0.02040816326530612, 'recall': 0.008403361344537815, 'f1': 0.011904761904761904, 'number': 119} | {'precision': 0.34987113402061853, 'recall': 0.5098591549295775, 'f1': 0.41497898356897206, 'number': 1065} | 0.2843            | 0.4466         | 0.3474     | 0.4803           |
+| 1.172         | 4.0   | 40   | 1.1080          | {'precision': 0.2609299097848716, 'recall': 0.4647713226205192, 'f1': 0.3342222222222222, 'number': 809}     | {'precision': 0.2, 'recall': 0.12605042016806722, 'f1': 0.15463917525773196, 'number': 119}                   | {'precision': 0.39096126255380204, 'recall': 0.5117370892018779, 'f1': 0.4432696217974787, 'number': 1065}  | 0.3216            | 0.4696         | 0.3818     | 0.5682           |
+| 1.0668        | 5.0   | 50   | 1.1224          | {'precision': 0.2859304084720121, 'recall': 0.4672435105067985, 'f1': 0.3547630220553731, 'number': 809}     | {'precision': 0.2571428571428571, 'recall': 0.15126050420168066, 'f1': 0.19047619047619044, 'number': 119}    | {'precision': 0.39935691318327976, 'recall': 0.5830985915492958, 'f1': 0.47404580152671755, 'number': 1065} | 0.3451            | 0.5103         | 0.4117     | 0.5719           |
+| 1.0053        | 6.0   | 60   | 1.0842          | {'precision': 0.31098430813124106, 'recall': 0.5389369592088998, 'f1': 0.3943916779737675, 'number': 809}    | {'precision': 0.3333333333333333, 'recall': 0.17647058823529413, 'f1': 0.23076923076923078, 'number': 119}    | {'precision': 0.4626998223801066, 'recall': 0.4892018779342723, 'f1': 0.4755819260611593, 'number': 1065}   | 0.3775            | 0.4907         | 0.4267     | 0.5869           |
+| 0.9367        | 7.0   | 70   | 1.0354          | {'precision': 0.33884297520661155, 'recall': 0.4561186650185414, 'f1': 0.38883034773445735, 'number': 809}   | {'precision': 0.27848101265822783, 'recall': 0.18487394957983194, 'f1': 0.2222222222222222, 'number': 119}    | {'precision': 0.4579100145137881, 'recall': 0.5924882629107981, 'f1': 0.5165779778960293, 'number': 1065}   | 0.4014            | 0.5128         | 0.4503     | 0.6069           |
+| 0.8736        | 8.0   | 80   | 1.0367          | {'precision': 0.3433583959899749, 'recall': 0.5080346106304079, 'f1': 0.4097706879361914, 'number': 809}     | {'precision': 0.24675324675324675, 'recall': 0.15966386554621848, 'f1': 0.19387755102040818, 'number': 119}   | {'precision': 0.4403292181069959, 'recall': 0.6028169014084507, 'f1': 0.5089179548156956, 'number': 1065}   | 0.3924            | 0.5379         | 0.4538     | 0.6083           |
+| 0.8322        | 9.0   | 90   | 1.0585          | {'precision': 0.38257575757575757, 'recall': 0.49938195302843014, 'f1': 0.43324396782841823, 'number': 809}  | {'precision': 0.1919191919191919, 'recall': 0.15966386554621848, 'f1': 0.17431192660550457, 'number': 119}    | {'precision': 0.48465266558966075, 'recall': 0.5633802816901409, 'f1': 0.5210594876248372, 'number': 1065}  | 0.4275            | 0.5133         | 0.4665     | 0.6171           |
+| 0.8201        | 10.0  | 100  | 1.0589          | {'precision': 0.3753527751646284, 'recall': 0.4932014833127318, 'f1': 0.42628205128205127, 'number': 809}    | {'precision': 0.275, 'recall': 0.18487394957983194, 'f1': 0.22110552763819097, 'number': 119}                 | {'precision': 0.4782945736434108, 'recall': 0.5793427230046948, 'f1': 0.5239915074309979, 'number': 1065}   | 0.4266            | 0.5208         | 0.4690     | 0.6086           |
+| 0.7451        | 11.0  | 110  | 1.0393          | {'precision': 0.3754716981132076, 'recall': 0.4919653893695921, 'f1': 0.42589620117710003, 'number': 809}    | {'precision': 0.2804878048780488, 'recall': 0.19327731092436976, 'f1': 0.22885572139303487, 'number': 119}    | {'precision': 0.4541832669322709, 'recall': 0.6422535211267606, 'f1': 0.5320886814469078, 'number': 1065}   | 0.4173            | 0.5544         | 0.4762     | 0.6132           |
+| 0.7445        | 12.0  | 120  | 1.0649          | {'precision': 0.3752166377816291, 'recall': 0.5352286773794809, 'f1': 0.4411614875191034, 'number': 809}     | {'precision': 0.2653061224489796, 'recall': 0.2184873949579832, 'f1': 0.23963133640552997, 'number': 119}     | {'precision': 0.49351701782820095, 'recall': 0.571830985915493, 'f1': 0.5297955632883862, 'number': 1065}   | 0.4296            | 0.5359         | 0.4769     | 0.6145           |
+| 0.7064        | 13.0  | 130  | 1.1267          | {'precision': 0.3775933609958506, 'recall': 0.5624227441285538, 'f1': 0.45183714001986097, 'number': 809}    | {'precision': 0.3116883116883117, 'recall': 0.20168067226890757, 'f1': 0.24489795918367344, 'number': 119}    | {'precision': 0.5072094995759118, 'recall': 0.5615023474178403, 'f1': 0.5329768270944741, 'number': 1065}   | 0.4376            | 0.5404         | 0.4836     | 0.6174           |
+| 0.6846        | 14.0  | 140  | 1.0692          | {'precision': 0.3945841392649903, 'recall': 0.5043263288009888, 'f1': 0.44275637547476937, 'number': 809}    | {'precision': 0.29411764705882354, 'recall': 0.21008403361344538, 'f1': 0.2450980392156863, 'number': 119}    | {'precision': 0.48787878787878786, 'recall': 0.6046948356807512, 'f1': 0.5400419287211741, 'number': 1065}  | 0.4416            | 0.5404         | 0.4860     | 0.6198           |
+| 0.6688        | 15.0  | 150  | 1.0890          | {'precision': 0.38420107719928187, 'recall': 0.5290482076637825, 'f1': 0.4451378055122205, 'number': 809}    | {'precision': 0.28888888888888886, 'recall': 0.2184873949579832, 'f1': 0.24880382775119617, 'number': 119}    | {'precision': 0.48959136468774095, 'recall': 0.596244131455399, 'f1': 0.5376799322607958, 'number': 1065}   | 0.4354            | 0.5464         | 0.4846     | 0.6258           |
+
+
+### Framework versions
+
+- Transformers 4.38.2
+- Pytorch 2.2.1+cu121
+- Datasets 2.18.0
+- Tokenizers 0.15.2

logs/events.out.tfevents.1710354675.1b7450536ea5.3929.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ae5d4a03f0a022e456182a04f7cb7a2baa95650842ad320b2c8a8d63610407aa
-size 13954
+oid sha256:ed45ace379cd3b2c42a150f3085bf538c0d0b213992c6c2d8cd3512250bd72a1
+size 15738

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c80528f7e8e20f073d0080133d253815d310f90b82e129ddcfb55bf2613de855
+oid sha256:c63f86ddd355554e93834574cb99b4194c966a4e922b37359297e68f301a748e
 size 450558212

preprocessor_config.json

@@ -0,0 +1,13 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,80 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}