End of training

README.md

@@ -1,3 +1,79 @@
 ---
-license: bsd-3-clause
+base_model: microsoft/layoutlm-base-uncased
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: docuAI
+  results: []
 ---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# docuAI
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6827
+- Answer: {'precision': 0.7172949002217295, 'recall': 0.799752781211372, 'f1': 0.7562828755113968, 'number': 809}
+- Header: {'precision': 0.33620689655172414, 'recall': 0.3277310924369748, 'f1': 0.33191489361702126, 'number': 119}
+- Question: {'precision': 0.7637931034482759, 'recall': 0.831924882629108, 'f1': 0.7964044943820225, 'number': 1065}
+- Overall Precision: 0.7218
+- Overall Recall: 0.7888
+- Overall F1: 0.7538
+- Overall Accuracy: 0.8097
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                      | Header                                                                                                        | Question                                                                                                   | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.7471        | 1.0   | 10   | 1.5418          | {'precision': 0.027965284474445518, 'recall': 0.03584672435105068, 'f1': 0.0314192849404117, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.26978723404255317, 'recall': 0.2976525821596244, 'f1': 0.2830357142857143, 'number': 1065} | 0.1564            | 0.1736         | 0.1646     | 0.4109           |
+| 1.3886        | 2.0   | 20   | 1.2047          | {'precision': 0.25207100591715975, 'recall': 0.26328800988875156, 'f1': 0.2575574365175332, 'number': 809}  | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.4723969252271139, 'recall': 0.6347417840375587, 'f1': 0.5416666666666667, 'number': 1065}  | 0.3906            | 0.4461         | 0.4165     | 0.5948           |
+| 1.0458        | 3.0   | 30   | 0.9213          | {'precision': 0.4836471754212091, 'recall': 0.6032138442521632, 'f1': 0.5368536853685368, 'number': 809}    | {'precision': 0.05263157894736842, 'recall': 0.008403361344537815, 'f1': 0.014492753623188406, 'number': 119} | {'precision': 0.6089795918367347, 'recall': 0.7004694835680751, 'f1': 0.651528384279476, 'number': 1065}   | 0.5482            | 0.6197         | 0.5817     | 0.7024           |
+| 0.8024        | 4.0   | 40   | 0.7873          | {'precision': 0.5814393939393939, 'recall': 0.7589616810877626, 'f1': 0.6584450402144773, 'number': 809}    | {'precision': 0.1044776119402985, 'recall': 0.058823529411764705, 'f1': 0.07526881720430108, 'number': 119}   | {'precision': 0.6488427773343974, 'recall': 0.7633802816901408, 'f1': 0.7014667817083693, 'number': 1065}  | 0.6035            | 0.7195         | 0.6564     | 0.7567           |
+| 0.6593        | 5.0   | 50   | 0.7148          | {'precision': 0.6419753086419753, 'recall': 0.7713226205191595, 'f1': 0.7007299270072992, 'number': 809}    | {'precision': 0.2602739726027397, 'recall': 0.15966386554621848, 'f1': 0.19791666666666666, 'number': 119}    | {'precision': 0.7365217391304347, 'recall': 0.7953051643192488, 'f1': 0.7647855530474039, 'number': 1065}  | 0.6788            | 0.7476         | 0.7116     | 0.7846           |
+| 0.5564        | 6.0   | 60   | 0.6806          | {'precision': 0.6945054945054945, 'recall': 0.7812113720642769, 'f1': 0.7353112274578244, 'number': 809}    | {'precision': 0.2647058823529412, 'recall': 0.226890756302521, 'f1': 0.24434389140271492, 'number': 119}      | {'precision': 0.7158067158067158, 'recall': 0.8206572769953052, 'f1': 0.7646544181977254, 'number': 1065}  | 0.6865            | 0.7692         | 0.7255     | 0.7947           |
+| 0.4838        | 7.0   | 70   | 0.6697          | {'precision': 0.6844396082698585, 'recall': 0.7775030902348579, 'f1': 0.7280092592592592, 'number': 809}    | {'precision': 0.2696629213483146, 'recall': 0.20168067226890757, 'f1': 0.23076923076923078, 'number': 119}    | {'precision': 0.738626964433416, 'recall': 0.8384976525821596, 'f1': 0.7854001759014952, 'number': 1065}   | 0.6973            | 0.7757         | 0.7344     | 0.8004           |
+| 0.4342        | 8.0   | 80   | 0.6709          | {'precision': 0.7062780269058296, 'recall': 0.7787391841779975, 'f1': 0.7407407407407407, 'number': 809}    | {'precision': 0.30097087378640774, 'recall': 0.2605042016806723, 'f1': 0.27927927927927926, 'number': 119}    | {'precision': 0.7415359207266722, 'recall': 0.8431924882629108, 'f1': 0.7891036906854131, 'number': 1065}  | 0.7067            | 0.7822         | 0.7426     | 0.8020           |
+| 0.386         | 9.0   | 90   | 0.6592          | {'precision': 0.7107258938244854, 'recall': 0.8108776266996292, 'f1': 0.7575057736720554, 'number': 809}    | {'precision': 0.2764227642276423, 'recall': 0.2857142857142857, 'f1': 0.2809917355371901, 'number': 119}      | {'precision': 0.75, 'recall': 0.8253521126760563, 'f1': 0.7858739383102369, 'number': 1065}                | 0.7074            | 0.7873         | 0.7452     | 0.8106           |
+| 0.3572        | 10.0  | 100  | 0.6611          | {'precision': 0.7080213903743315, 'recall': 0.8182941903584673, 'f1': 0.7591743119266056, 'number': 809}    | {'precision': 0.29906542056074764, 'recall': 0.2689075630252101, 'f1': 0.28318584070796454, 'number': 119}    | {'precision': 0.7532133676092545, 'recall': 0.8253521126760563, 'f1': 0.7876344086021505, 'number': 1065}  | 0.7121            | 0.7893         | 0.7487     | 0.8102           |
+| 0.3264        | 11.0  | 110  | 0.6828          | {'precision': 0.7325056433408578, 'recall': 0.8022249690976514, 'f1': 0.7657817109144542, 'number': 809}    | {'precision': 0.3, 'recall': 0.3025210084033613, 'f1': 0.301255230125523, 'number': 119}                      | {'precision': 0.7525773195876289, 'recall': 0.8225352112676056, 'f1': 0.7860026917900403, 'number': 1065}  | 0.7194            | 0.7832         | 0.7499     | 0.8055           |
+| 0.3132        | 12.0  | 120  | 0.6722          | {'precision': 0.7123893805309734, 'recall': 0.796044499381953, 'f1': 0.7518972562755399, 'number': 809}     | {'precision': 0.3391304347826087, 'recall': 0.3277310924369748, 'f1': 0.3333333333333333, 'number': 119}      | {'precision': 0.7548605240912933, 'recall': 0.8384976525821596, 'f1': 0.7944839857651246, 'number': 1065}  | 0.7157            | 0.7908         | 0.7514     | 0.8082           |
+| 0.293         | 13.0  | 130  | 0.6817          | {'precision': 0.7109634551495017, 'recall': 0.7935723114956736, 'f1': 0.75, 'number': 809}                  | {'precision': 0.3277310924369748, 'recall': 0.3277310924369748, 'f1': 0.3277310924369748, 'number': 119}      | {'precision': 0.7680776014109347, 'recall': 0.8178403755868544, 'f1': 0.7921782628467485, 'number': 1065}  | 0.7199            | 0.7787         | 0.7481     | 0.8078           |
+| 0.282         | 14.0  | 140  | 0.6845          | {'precision': 0.712707182320442, 'recall': 0.7972805933250927, 'f1': 0.7526254375729288, 'number': 809}     | {'precision': 0.3333333333333333, 'recall': 0.3277310924369748, 'f1': 0.3305084745762712, 'number': 119}      | {'precision': 0.768624014022787, 'recall': 0.8234741784037559, 'f1': 0.7951042611060744, 'number': 1065}   | 0.7217            | 0.7832         | 0.7512     | 0.8094           |
+| 0.2728        | 15.0  | 150  | 0.6827          | {'precision': 0.7172949002217295, 'recall': 0.799752781211372, 'f1': 0.7562828755113968, 'number': 809}     | {'precision': 0.33620689655172414, 'recall': 0.3277310924369748, 'f1': 0.33191489361702126, 'number': 119}    | {'precision': 0.7637931034482759, 'recall': 0.831924882629108, 'f1': 0.7964044943820225, 'number': 1065}   | 0.7218            | 0.7888         | 0.7538     | 0.8097           |
+
+
+### Framework versions
+
+- Transformers 4.33.2
+- Pytorch 2.0.1+cu118
+- Datasets 2.14.5
+- Tokenizers 0.13.3

preprocessor_config.json

@@ -0,0 +1,14 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "feature_extractor_type": "LayoutLMv2FeatureExtractor",
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:98467f71c4b2d1ed50b31bb79b79479f70833112d7c6b2b6a2a4f5bfb3aa0fa2
+oid sha256:99ce6837ab1ab156c49421293d5fb0832718d01409db1d10a4c1a14b0a8dd05d
 size 450603969

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,38 @@
+{
+  "additional_special_tokens": null,
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}