Hugging Face's logo

tiennvcs
/
bert-large-uncased-finetuned-docvqa

Question Answering
Transformers
PyTorch
TensorBoard
bert
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
tiennvcs commited on
Commit
8bf070f
路
1 Parent(s): 94b0c55

Fine tuning BERT large for DocVQA

Browse files
Files changed (11) hide show
  1. .gitignore +1 -0
  2. README.md +173 -0
  3. config.json +26 -0
  4. pytorch_model.bin +3 -0
  5. runs/Oct22_23-35-50_ae94a4e6c92d/1634945767.1380427/events.out.tfevents.1634945767.ae94a4e6c92d.482.1 +3 -0
  6. runs/Oct22_23-35-50_ae94a4e6c92d/events.out.tfevents.1634945767.ae94a4e6c92d.482.0 +3 -0
  7. special_tokens_map.json +1 -0
  8. tokenizer.json +0 -0
  9. tokenizer_config.json +1 -0
  10. training_args.bin +3 -0
  11. vocab.txt +0 -0
.gitignore ADDED
@@ -0,0 +1 @@
 
 
1
+ checkpoint-*/
README.md ADDED
@@ -0,0 +1,173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: bert-large-uncased-finetuned-docvqa
7
+ results:
8
+ - task:
9
+ name: Question Answering
10
+ type: question-answering
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # bert-large-uncased-finetuned-docvqa
17
+
18
+ This model is a fine-tuned version of [bert-large-uncased](https://huggingface.co/bert-large-uncased) on an unknown dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 3.6367
21
+
22
+ ## Model description
23
+
24
+ More information needed
25
+
26
+ ## Intended uses & limitations
27
+
28
+ More information needed
29
+
30
+ ## Training and evaluation data
31
+
32
+ More information needed
33
+
34
+ ## Training procedure
35
+
36
+ ### Training hyperparameters
37
+
38
+ The following hyperparameters were used during training:
39
+ - learning_rate: 2e-05
40
+ - train_batch_size: 2
41
+ - eval_batch_size: 2
42
+ - seed: 250500
43
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
+ - lr_scheduler_type: linear
45
+ - num_epochs: 6
46
+
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:------:|:---------------:|
51
+ | 2.5228 | 0.05 | 1000 | 2.6645 |
52
+ | 2.4909 | 0.1 | 2000 | 2.8985 |
53
+ | 2.1679 | 0.16 | 3000 | 2.3551 |
54
+ | 1.9451 | 0.21 | 4000 | 2.2226 |
55
+ | 1.6814 | 0.26 | 5000 | 2.1590 |
56
+ | 1.8868 | 0.31 | 6000 | 2.6197 |
57
+ | 1.6618 | 0.36 | 7000 | 2.3632 |
58
+ | 1.8313 | 0.41 | 8000 | 2.4519 |
59
+ | 1.7017 | 0.47 | 9000 | 2.2682 |
60
+ | 1.8169 | 0.52 | 10000 | 2.4486 |
61
+ | 1.7074 | 0.57 | 11000 | 2.3862 |
62
+ | 1.7674 | 0.62 | 12000 | 2.1801 |
63
+ | 1.8134 | 0.67 | 13000 | 2.3032 |
64
+ | 1.8334 | 0.73 | 14000 | 2.4205 |
65
+ | 1.6819 | 0.78 | 15000 | 2.2398 |
66
+ | 1.5846 | 0.83 | 16000 | 2.3834 |
67
+ | 1.6758 | 0.88 | 17000 | 1.9683 |
68
+ | 1.6303 | 0.93 | 18000 | 2.3297 |
69
+ | 1.5652 | 0.98 | 19000 | 2.0581 |
70
+ | 1.3045 | 1.04 | 20000 | 2.4950 |
71
+ | 1.2393 | 1.09 | 21000 | 2.6622 |
72
+ | 1.1526 | 1.14 | 22000 | 2.3749 |
73
+ | 1.2631 | 1.19 | 23000 | 2.3915 |
74
+ | 1.1846 | 1.24 | 24000 | 2.2592 |
75
+ | 1.2731 | 1.3 | 25000 | 2.4239 |
76
+ | 1.3057 | 1.35 | 26000 | 2.2920 |
77
+ | 1.134 | 1.4 | 27000 | 2.3107 |
78
+ | 1.2017 | 1.45 | 28000 | 2.4271 |
79
+ | 1.2202 | 1.5 | 29000 | 2.1814 |
80
+ | 1.2179 | 1.56 | 30000 | 2.3365 |
81
+ | 1.2359 | 1.61 | 31000 | 2.1256 |
82
+ | 1.1964 | 1.66 | 32000 | 2.1720 |
83
+ | 1.269 | 1.71 | 33000 | 2.4363 |
84
+ | 1.1812 | 1.76 | 34000 | 2.2372 |
85
+ | 1.2187 | 1.81 | 35000 | 2.2318 |
86
+ | 1.1805 | 1.87 | 36000 | 2.3693 |
87
+ | 1.1458 | 1.92 | 37000 | 2.5128 |
88
+ | 1.1958 | 1.97 | 38000 | 2.1311 |
89
+ | 0.8924 | 2.02 | 39000 | 2.4635 |
90
+ | 0.869 | 2.07 | 40000 | 2.8231 |
91
+ | 0.8333 | 2.13 | 41000 | 2.6762 |
92
+ | 0.9194 | 2.18 | 42000 | 2.4588 |
93
+ | 0.8089 | 2.23 | 43000 | 2.6443 |
94
+ | 0.8612 | 2.28 | 44000 | 2.4300 |
95
+ | 0.7981 | 2.33 | 45000 | 2.7418 |
96
+ | 0.9765 | 2.38 | 46000 | 2.6543 |
97
+ | 0.8646 | 2.44 | 47000 | 2.5990 |
98
+ | 1.0316 | 2.49 | 48000 | 2.4625 |
99
+ | 0.9862 | 2.54 | 49000 | 2.4691 |
100
+ | 1.027 | 2.59 | 50000 | 2.4156 |
101
+ | 0.9412 | 2.64 | 51000 | 2.4204 |
102
+ | 0.9353 | 2.7 | 52000 | 2.4933 |
103
+ | 0.9509 | 2.75 | 53000 | 2.4708 |
104
+ | 0.9351 | 2.8 | 54000 | 2.5351 |
105
+ | 0.9968 | 2.85 | 55000 | 2.2506 |
106
+ | 1.025 | 2.9 | 56000 | 2.6317 |
107
+ | 1.627 | 2.95 | 57000 | 2.7843 |
108
+ | 0.9294 | 3.01 | 58000 | 2.9396 |
109
+ | 0.6043 | 3.06 | 59000 | 3.1560 |
110
+ | 0.7903 | 3.11 | 60000 | 2.8330 |
111
+ | 0.7373 | 3.16 | 61000 | 2.9422 |
112
+ | 0.6499 | 3.21 | 62000 | 3.0948 |
113
+ | 0.6411 | 3.27 | 63000 | 2.7900 |
114
+ | 0.625 | 3.32 | 64000 | 2.5268 |
115
+ | 0.6264 | 3.37 | 65000 | 2.8701 |
116
+ | 0.6143 | 3.42 | 66000 | 3.2544 |
117
+ | 0.6286 | 3.47 | 67000 | 2.6208 |
118
+ | 0.739 | 3.53 | 68000 | 2.8107 |
119
+ | 0.5981 | 3.58 | 69000 | 2.8073 |
120
+ | 0.6502 | 3.63 | 70000 | 2.6293 |
121
+ | 0.6548 | 3.68 | 71000 | 2.9501 |
122
+ | 0.7243 | 3.73 | 72000 | 2.7917 |
123
+ | 0.598 | 3.78 | 73000 | 2.9341 |
124
+ | 0.6159 | 3.84 | 74000 | 2.7629 |
125
+ | 0.5905 | 3.89 | 75000 | 2.6441 |
126
+ | 0.6393 | 3.94 | 76000 | 2.6660 |
127
+ | 0.677 | 3.99 | 77000 | 2.7616 |
128
+ | 0.3281 | 4.04 | 78000 | 3.6873 |
129
+ | 0.4524 | 4.1 | 79000 | 3.3441 |
130
+ | 0.3994 | 4.15 | 80000 | 3.3129 |
131
+ | 0.4686 | 4.2 | 81000 | 3.1813 |
132
+ | 0.5293 | 4.25 | 82000 | 2.9088 |
133
+ | 0.3961 | 4.3 | 83000 | 3.0765 |
134
+ | 0.4406 | 4.35 | 84000 | 3.1254 |
135
+ | 0.401 | 4.41 | 85000 | 3.2415 |
136
+ | 0.4594 | 4.46 | 86000 | 3.0691 |
137
+ | 0.4523 | 4.51 | 87000 | 3.0493 |
138
+ | 0.4719 | 4.56 | 88000 | 3.1352 |
139
+ | 0.4895 | 4.61 | 89000 | 2.8991 |
140
+ | 0.423 | 4.67 | 90000 | 3.1738 |
141
+ | 0.3984 | 4.72 | 91000 | 3.1862 |
142
+ | 0.4206 | 4.77 | 92000 | 3.1213 |
143
+ | 0.4587 | 4.82 | 93000 | 3.0030 |
144
+ | 0.381 | 4.87 | 94000 | 3.3218 |
145
+ | 0.4138 | 4.92 | 95000 | 3.1529 |
146
+ | 0.4003 | 4.98 | 96000 | 3.1375 |
147
+ | 0.2098 | 5.03 | 97000 | 3.7443 |
148
+ | 0.2334 | 5.08 | 98000 | 3.7359 |
149
+ | 0.2534 | 5.13 | 99000 | 3.7814 |
150
+ | 0.3067 | 5.18 | 100000 | 3.7128 |
151
+ | 0.2363 | 5.24 | 101000 | 3.6091 |
152
+ | 0.2652 | 5.29 | 102000 | 3.4015 |
153
+ | 0.3311 | 5.34 | 103000 | 3.4793 |
154
+ | 0.2344 | 5.39 | 104000 | 3.6792 |
155
+ | 0.2741 | 5.44 | 105000 | 3.5385 |
156
+ | 0.2896 | 5.5 | 106000 | 3.8118 |
157
+ | 0.2071 | 5.55 | 107000 | 3.8690 |
158
+ | 0.3023 | 5.6 | 108000 | 3.7087 |
159
+ | 0.3299 | 5.65 | 109000 | 3.4925 |
160
+ | 0.1943 | 5.7 | 110000 | 3.6739 |
161
+ | 0.2488 | 5.75 | 111000 | 3.7614 |
162
+ | 0.3138 | 5.81 | 112000 | 3.5156 |
163
+ | 0.2555 | 5.86 | 113000 | 3.6056 |
164
+ | 0.2918 | 5.91 | 114000 | 3.6533 |
165
+ | 0.2751 | 5.96 | 115000 | 3.6367 |
166
+
167
+
168
+ ### Framework versions
169
+
170
+ - Transformers 4.10.0
171
+ - Pytorch 1.8.0+cu101
172
+ - Datasets 1.11.0
173
+ - Tokenizers 0.10.3
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-large-uncased",
3
+ "architectures": [
4
+ "BertForQuestionAnswering"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 1024,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 4096,
14
+ "layer_norm_eps": 1e-12,
15
+ "max_position_embeddings": 512,
16
+ "model_type": "bert",
17
+ "num_attention_heads": 16,
18
+ "num_hidden_layers": 24,
19
+ "pad_token_id": 0,
20
+ "position_embedding_type": "absolute",
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.10.0",
23
+ "type_vocab_size": 2,
24
+ "use_cache": true,
25
+ "vocab_size": 30522
26
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aeca030632bd5377f0f061fad5feaa9b4269b85e28f1f57e118cbe06b16a3acc
3
+ size 1336547639
runs/Oct22_23-35-50_ae94a4e6c92d/1634945767.1380427/events.out.tfevents.1634945767.ae94a4e6c92d.482.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:120ad5663d8ca32c27221168c3b09d82e8ea083f1fd1bbe2fc559d421a39cef4
3
+ size 4280
runs/Oct22_23-35-50_ae94a4e6c92d/events.out.tfevents.1634945767.ae94a4e6c92d.482.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a87eaf7089f1fe8d377600d37a4b547f9c851eafd3d2480899553362ee299c7
3
+ size 71927
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "model_max_length": 512, "special_tokens_map_file": null, "name_or_path": "bert-large-uncased", "tokenizer_class": "BertTokenizer"}
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:948487426d29059c701946a482907168a565044191ccc9a1cf62df4911e51d71
3
+ size 2671
vocab.txt ADDED
The diff for this file is too large to render. See raw diff