End of training

README.md

@@ -0,0 +1,80 @@
+---
+base_model: microsoft/layoutlm-base-uncased
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6565
+- Answer: {'precision': 0.7244897959183674, 'recall': 0.7898640296662547, 'f1': 0.7557658190419871, 'number': 809}
+- Header: {'precision': 0.34710743801652894, 'recall': 0.35294117647058826, 'f1': 0.35000000000000003, 'number': 119}
+- Question: {'precision': 0.7786458333333334, 'recall': 0.8422535211267606, 'f1': 0.8092016238159676, 'number': 1065}
+- Overall Precision: 0.7323
+- Overall Recall: 0.7918
+- Overall F1: 0.7608
+- Overall Accuracy: 0.8097
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                       | Header                                                                                                        | Question                                                                                                     | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.7236        | 1.0   | 10   | 1.5431          | {'precision': 0.03571428571428571, 'recall': 0.029666254635352288, 'f1': 0.03241053342336259, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.37149532710280375, 'recall': 0.29859154929577464, 'f1': 0.33107756376887043, 'number': 1065} | 0.2238            | 0.1716         | 0.1943     | 0.3796           |
+| 1.3695        | 2.0   | 20   | 1.1737          | {'precision': 0.2528032619775739, 'recall': 0.3065512978986403, 'f1': 0.2770949720670391, 'number': 809}     | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                   | {'precision': 0.4624819624819625, 'recall': 0.6018779342723005, 'f1': 0.5230518155854754, 'number': 1065}    | 0.3748            | 0.4461         | 0.4073     | 0.6106           |
+| 1.0404        | 3.0   | 30   | 0.9013          | {'precision': 0.5138613861386139, 'recall': 0.6415327564894932, 'f1': 0.5706432105552502, 'number': 809}     | {'precision': 0.07894736842105263, 'recall': 0.025210084033613446, 'f1': 0.038216560509554146, 'number': 119} | {'precision': 0.5854601701469451, 'recall': 0.7107981220657277, 'f1': 0.642069550466497, 'number': 1065}     | 0.5463            | 0.6417         | 0.5902     | 0.7217           |
+| 0.8081        | 4.0   | 40   | 0.7592          | {'precision': 0.5993914807302231, 'recall': 0.73053152039555, 'f1': 0.6584958217270194, 'number': 809}       | {'precision': 0.16417910447761194, 'recall': 0.09243697478991597, 'f1': 0.1182795698924731, 'number': 119}    | {'precision': 0.6539379474940334, 'recall': 0.7718309859154929, 'f1': 0.7080103359173127, 'number': 1065}    | 0.6165            | 0.7145         | 0.6619     | 0.7633           |
+| 0.6544        | 5.0   | 50   | 0.6873          | {'precision': 0.6234817813765182, 'recall': 0.761433868974042, 'f1': 0.6855870895937674, 'number': 809}      | {'precision': 0.2972972972972973, 'recall': 0.18487394957983194, 'f1': 0.22797927461139897, 'number': 119}    | {'precision': 0.7099236641221374, 'recall': 0.7859154929577464, 'f1': 0.7459893048128342, 'number': 1065}    | 0.6582            | 0.7401         | 0.6967     | 0.7795           |
+| 0.5597        | 6.0   | 60   | 0.6540          | {'precision': 0.674217907227616, 'recall': 0.7725587144622992, 'f1': 0.7200460829493088, 'number': 809}      | {'precision': 0.3157894736842105, 'recall': 0.20168067226890757, 'f1': 0.24615384615384614, 'number': 119}    | {'precision': 0.7269681742043551, 'recall': 0.8150234741784037, 'f1': 0.7684816290393979, 'number': 1065}    | 0.6905            | 0.7612         | 0.7241     | 0.7894           |
+| 0.4916        | 7.0   | 70   | 0.6434          | {'precision': 0.6870229007633588, 'recall': 0.7787391841779975, 'f1': 0.7300115874855155, 'number': 809}     | {'precision': 0.31683168316831684, 'recall': 0.2689075630252101, 'f1': 0.29090909090909095, 'number': 119}    | {'precision': 0.7291311754684838, 'recall': 0.8037558685446009, 'f1': 0.7646270656543098, 'number': 1065}    | 0.6925            | 0.7617         | 0.7254     | 0.7949           |
+| 0.4415        | 8.0   | 80   | 0.6266          | {'precision': 0.7018498367791077, 'recall': 0.7972805933250927, 'f1': 0.7465277777777779, 'number': 809}     | {'precision': 0.3090909090909091, 'recall': 0.2857142857142857, 'f1': 0.296943231441048, 'number': 119}       | {'precision': 0.7609630266552021, 'recall': 0.8309859154929577, 'f1': 0.7944344703770198, 'number': 1065}    | 0.7135            | 0.7847         | 0.7474     | 0.8045           |
+| 0.3702        | 9.0   | 90   | 0.6265          | {'precision': 0.706858407079646, 'recall': 0.7898640296662547, 'f1': 0.7460595446584939, 'number': 809}      | {'precision': 0.3786407766990291, 'recall': 0.3277310924369748, 'f1': 0.35135135135135137, 'number': 119}     | {'precision': 0.7695614789337919, 'recall': 0.8403755868544601, 'f1': 0.8034111310592459, 'number': 1065}    | 0.7249            | 0.7893         | 0.7557     | 0.8026           |
+| 0.341         | 10.0  | 100  | 0.6384          | {'precision': 0.7091319052987599, 'recall': 0.7775030902348579, 'f1': 0.741745283018868, 'number': 809}      | {'precision': 0.37, 'recall': 0.31092436974789917, 'f1': 0.3378995433789954, 'number': 119}                   | {'precision': 0.7773000859845228, 'recall': 0.8488262910798122, 'f1': 0.8114901256732495, 'number': 1065}    | 0.7302            | 0.7878         | 0.7579     | 0.8042           |
+| 0.3141        | 11.0  | 110  | 0.6472          | {'precision': 0.7158962795941376, 'recall': 0.7849196538936959, 'f1': 0.7488207547169812, 'number': 809}     | {'precision': 0.34782608695652173, 'recall': 0.33613445378151263, 'f1': 0.3418803418803419, 'number': 119}    | {'precision': 0.7785467128027682, 'recall': 0.8450704225352113, 'f1': 0.810445745159838, 'number': 1065}     | 0.7298            | 0.7903         | 0.7589     | 0.8054           |
+| 0.2951        | 12.0  | 120  | 0.6467          | {'precision': 0.7165532879818595, 'recall': 0.7812113720642769, 'f1': 0.7474866942637493, 'number': 809}     | {'precision': 0.34959349593495936, 'recall': 0.36134453781512604, 'f1': 0.35537190082644626, 'number': 119}   | {'precision': 0.7713546160483176, 'recall': 0.8394366197183099, 'f1': 0.8039568345323741, 'number': 1065}    | 0.7250            | 0.7873         | 0.7549     | 0.8095           |
+| 0.2803        | 13.0  | 130  | 0.6506          | {'precision': 0.7177777777777777, 'recall': 0.7985166872682324, 'f1': 0.7559976594499708, 'number': 809}     | {'precision': 0.35537190082644626, 'recall': 0.36134453781512604, 'f1': 0.3583333333333333, 'number': 119}    | {'precision': 0.7685738684884714, 'recall': 0.8450704225352113, 'f1': 0.8050089445438282, 'number': 1065}    | 0.7249            | 0.7973         | 0.7594     | 0.8049           |
+| 0.2623        | 14.0  | 140  | 0.6554          | {'precision': 0.7228506787330317, 'recall': 0.7898640296662547, 'f1': 0.754873006497342, 'number': 809}      | {'precision': 0.3559322033898305, 'recall': 0.35294117647058826, 'f1': 0.35443037974683544, 'number': 119}    | {'precision': 0.7793223284100782, 'recall': 0.8422535211267606, 'f1': 0.8095667870036102, 'number': 1065}    | 0.7329            | 0.7918         | 0.7612     | 0.8102           |
+| 0.2699        | 15.0  | 150  | 0.6565          | {'precision': 0.7244897959183674, 'recall': 0.7898640296662547, 'f1': 0.7557658190419871, 'number': 809}     | {'precision': 0.34710743801652894, 'recall': 0.35294117647058826, 'f1': 0.35000000000000003, 'number': 119}   | {'precision': 0.7786458333333334, 'recall': 0.8422535211267606, 'f1': 0.8092016238159676, 'number': 1065}    | 0.7323            | 0.7918         | 0.7608     | 0.8097           |
+
+
+### Framework versions
+
+- Transformers 4.35.2
+- Pytorch 2.1.0+cu118
+- Datasets 2.15.0
+- Tokenizers 0.15.0

logs/events.out.tfevents.1701496677.c0c31398db08.6950.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:943e0f1fa20174317dbfbaeb9f395b9b188f592065899943ea405b06d7d8cdb7
-size 12912
+oid sha256:af62127b53e755efe0131eacc604e216a7441a697b9dbd6a714f748b97488352
+size 14588

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e7df75f5d24ab812e7fa86d12accf2c67455d4c219529b6ec31e726e0e6abd0a
+oid sha256:c95a6d7e0080dfce8dd05242ddcb196ca40d8998e75278806aaefa585fe6f2a1
 size 450558212

preprocessor_config.json

@@ -0,0 +1,14 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "feature_extractor_type": "LayoutLMv2FeatureExtractor",
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,80 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}