mdizak commited on
Commit
786494d
1 Parent(s): 944d302
README.md CHANGED
@@ -1,3 +1,152 @@
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language: en
3
+ datasets:
4
+ - conll2003
5
+ license: mit
6
+ model-index:
7
+ - name: dslim/bert-large-NER
8
+ results:
9
+ - task:
10
+ type: token-classification
11
+ name: Token Classification
12
+ dataset:
13
+ name: conll2003
14
+ type: conll2003
15
+ config: conll2003
16
+ split: test
17
+ metrics:
18
+ - name: Accuracy
19
+ type: accuracy
20
+ value: 0.9031688753722759
21
+ verified: true
22
+ - name: Precision
23
+ type: precision
24
+ value: 0.920025068328604
25
+ verified: true
26
+ - name: Recall
27
+ type: recall
28
+ value: 0.9193688678588825
29
+ verified: true
30
+ - name: F1
31
+ type: f1
32
+ value: 0.9196968510445761
33
+ verified: true
34
+ - name: loss
35
+ type: loss
36
+ value: 0.5085050463676453
37
+ verified: true
38
  ---
39
+ # bert-large-NER
40
+
41
+ ## Model description
42
+
43
+ **bert-large-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).
44
+
45
+ Specifically, this model is a *bert-large-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.
46
+
47
+ If you'd like to use a smaller BERT model fine-tuned on the same dataset, a [**bert-base-NER**](https://huggingface.co/dslim/bert-base-NER/) version is also available.
48
+
49
+
50
+ ## Intended uses & limitations
51
+
52
+ #### How to use
53
+
54
+ You can use this model with Transformers *pipeline* for NER.
55
+
56
+ ```python
57
+ from transformers import AutoTokenizer, AutoModelForTokenClassification
58
+ from transformers import pipeline
59
+
60
+ tokenizer = AutoTokenizer.from_pretrained("dslim/bert-large-NER")
61
+ model = AutoModelForTokenClassification.from_pretrained("dslim/bert-large-NER")
62
+
63
+ nlp = pipeline("ner", model=model, tokenizer=tokenizer)
64
+ example = "My name is Wolfgang and I live in Berlin"
65
+
66
+ ner_results = nlp(example)
67
+ print(ner_results)
68
+ ```
69
+
70
+ #### Limitations and bias
71
+
72
+ This model is limited by its training dataset of entity-annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains. Furthermore, the model occassionally tags subword tokens as entities and post-processing of results may be necessary to handle those cases.
73
+
74
+ ## Training data
75
+
76
+ This model was fine-tuned on English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.
77
+
78
+ The training dataset distinguishes between the beginning and continuation of an entity so that if there are back-to-back entities of the same type, the model can output where the second entity begins. As in the dataset, each token will be classified as one of the following classes:
79
+
80
+ Abbreviation|Description
81
+ -|-
82
+ O|Outside of a named entity
83
+ B-MIS |Beginning of a miscellaneous entity right after another miscellaneous entity
84
+ I-MIS | Miscellaneous entity
85
+ B-PER |Beginning of a person’s name right after another person’s name
86
+ I-PER |Person’s name
87
+ B-ORG |Beginning of an organization right after another organization
88
+ I-ORG |organization
89
+ B-LOC |Beginning of a location right after another location
90
+ I-LOC |Location
91
+
92
+
93
+ ### CoNLL-2003 English Dataset Statistics
94
+ This dataset was derived from the Reuters corpus which consists of Reuters news stories. You can read more about how this dataset was created in the CoNLL-2003 paper.
95
+ #### # of training examples per entity type
96
+ Dataset|LOC|MISC|ORG|PER
97
+ -|-|-|-|-
98
+ Train|7140|3438|6321|6600
99
+ Dev|1837|922|1341|1842
100
+ Test|1668|702|1661|1617
101
+ #### # of articles/sentences/tokens per dataset
102
+ Dataset |Articles |Sentences |Tokens
103
+ -|-|-|-
104
+ Train |946 |14,987 |203,621
105
+ Dev |216 |3,466 |51,362
106
+ Test |231 |3,684 |46,435
107
+
108
+ ## Training procedure
109
+
110
+ This model was trained on a single NVIDIA V100 GPU with recommended hyperparameters from the [original BERT paper](https://arxiv.org/pdf/1810.04805) which trained & evaluated the model on CoNLL-2003 NER task.
111
+
112
+ ## Eval results
113
+ metric|dev|test
114
+ -|-|-
115
+ f1 |95.7 |91.7
116
+ precision |95.3 |91.2
117
+ recall |96.1 |92.3
118
+
119
+ The test metrics are a little lower than the official Google BERT results which encoded document context & experimented with CRF. More on replicating the original results [here](https://github.com/google-research/bert/issues/223).
120
+
121
+ ### BibTeX entry and citation info
122
+
123
+ ```
124
+ @article{DBLP:journals/corr/abs-1810-04805,
125
+ author = {Jacob Devlin and
126
+ Ming{-}Wei Chang and
127
+ Kenton Lee and
128
+ Kristina Toutanova},
129
+ title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
130
+ Understanding},
131
+ journal = {CoRR},
132
+ volume = {abs/1810.04805},
133
+ year = {2018},
134
+ url = {http://arxiv.org/abs/1810.04805},
135
+ archivePrefix = {arXiv},
136
+ eprint = {1810.04805},
137
+ timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
138
+ biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
139
+ bibsource = {dblp computer science bibliography, https://dblp.org}
140
+ }
141
+ ```
142
+ ```
143
+ @inproceedings{tjong-kim-sang-de-meulder-2003-introduction,
144
+ title = "Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition",
145
+ author = "Tjong Kim Sang, Erik F. and
146
+ De Meulder, Fien",
147
+ booktitle = "Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003",
148
+ year = "2003",
149
+ url = "https://www.aclweb.org/anthology/W03-0419",
150
+ pages = "142--147",
151
+ }
152
+ ```
bert-large-NER-rust/.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
bert-large-NER-rust/README.md ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
config.json ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "BertForTokenClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.1,
6
+ "bos_token_id": null,
7
+ "directionality": "bidi",
8
+ "do_sample": false,
9
+ "eos_token_ids": null,
10
+ "finetuning_task": null,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 1024,
14
+ "id2label": {
15
+ "0": "O",
16
+ "1": "B-MISC",
17
+ "2": "I-MISC",
18
+ "3": "B-PER",
19
+ "4": "I-PER",
20
+ "5": "B-ORG",
21
+ "6": "I-ORG",
22
+ "7": "B-LOC",
23
+ "8": "I-LOC"
24
+ },
25
+ "initializer_range": 0.02,
26
+ "intermediate_size": 4096,
27
+ "is_decoder": false,
28
+ "label2id": {
29
+ "B-LOC": 7,
30
+ "B-MISC": 1,
31
+ "B-ORG": 5,
32
+ "B-PER": 3,
33
+ "I-LOC": 8,
34
+ "I-MISC": 2,
35
+ "I-ORG": 6,
36
+ "I-PER": 4,
37
+ "O": 0
38
+ },
39
+ "layer_norm_eps": 1e-12,
40
+ "length_penalty": 1.0,
41
+ "max_length": 20,
42
+ "max_position_embeddings": 512,
43
+ "model_type": "bert",
44
+ "num_attention_heads": 16,
45
+ "num_beams": 1,
46
+ "num_hidden_layers": 24,
47
+ "num_labels": 9,
48
+ "num_return_sequences": 1,
49
+ "output_attentions": false,
50
+ "output_hidden_states": false,
51
+ "output_past": true,
52
+ "pad_token_id": 0,
53
+ "pooler_fc_size": 768,
54
+ "pooler_num_attention_heads": 12,
55
+ "pooler_num_fc_layers": 3,
56
+ "pooler_size_per_head": 128,
57
+ "pooler_type": "first_token_transform",
58
+ "pruned_heads": {},
59
+ "repetition_penalty": 1.0,
60
+ "temperature": 1.0,
61
+ "top_k": 50,
62
+ "top_p": 1.0,
63
+ "torchscript": false,
64
+ "type_vocab_size": 2,
65
+ "use_bfloat16": false,
66
+ "vocab_size": 28996
67
+ }
flax_model.msgpack ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d63d39520929bbeab16344157582be46f09a3ab43095b14f1b1586636a50f171
3
+ size 1330169528
model.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d627d10676d2ee3e2a5f7a91605f1527b702e2c618608fce8aaf6a0310514d5d
3
+ size 1334481458
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:81d6a7a3967f136e18bac045a2c8c0ab30f39ae70268129e60255afa21d4205b
3
+ size 1334400964
onnx/added_tokens.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "[CLS]": 101,
3
+ "[MASK]": 103,
4
+ "[PAD]": 0,
5
+ "[SEP]": 102,
6
+ "[UNK]": 100
7
+ }
onnx/config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "dslim/bert-large-NER",
3
+ "architectures": [
4
+ "BertForTokenClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "directionality": "bidi",
9
+ "eos_token_ids": null,
10
+ "hidden_act": "gelu",
11
+ "hidden_dropout_prob": 0.1,
12
+ "hidden_size": 1024,
13
+ "id2label": {
14
+ "0": "O",
15
+ "1": "B-MISC",
16
+ "2": "I-MISC",
17
+ "3": "B-PER",
18
+ "4": "I-PER",
19
+ "5": "B-ORG",
20
+ "6": "I-ORG",
21
+ "7": "B-LOC",
22
+ "8": "I-LOC"
23
+ },
24
+ "initializer_range": 0.02,
25
+ "intermediate_size": 4096,
26
+ "label2id": {
27
+ "B-LOC": 7,
28
+ "B-MISC": 1,
29
+ "B-ORG": 5,
30
+ "B-PER": 3,
31
+ "I-LOC": 8,
32
+ "I-MISC": 2,
33
+ "I-ORG": 6,
34
+ "I-PER": 4,
35
+ "O": 0
36
+ },
37
+ "layer_norm_eps": 1e-12,
38
+ "max_position_embeddings": 512,
39
+ "model_type": "bert",
40
+ "num_attention_heads": 16,
41
+ "num_hidden_layers": 24,
42
+ "output_past": true,
43
+ "pad_token_id": 0,
44
+ "pooler_fc_size": 768,
45
+ "pooler_num_attention_heads": 12,
46
+ "pooler_num_fc_layers": 3,
47
+ "pooler_size_per_head": 128,
48
+ "pooler_type": "first_token_transform",
49
+ "position_embedding_type": "absolute",
50
+ "transformers_version": "4.34.0",
51
+ "type_vocab_size": 2,
52
+ "use_cache": true,
53
+ "vocab_size": 28996
54
+ }
onnx/model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:46ba1e057362a10e8163f69a2c9ef1cbb609e9eca6ab265835d8f7fa4c44106b
3
+ size 1330682413
onnx/special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
onnx/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
onnx/tokenizer_config.json ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "additional_special_tokens": [],
45
+ "clean_up_tokenization_spaces": true,
46
+ "cls_token": "[CLS]",
47
+ "do_basic_tokenize": true,
48
+ "do_lower_case": false,
49
+ "mask_token": "[MASK]",
50
+ "max_len": 512,
51
+ "model_max_length": 512,
52
+ "never_split": null,
53
+ "pad_token": "[PAD]",
54
+ "sep_token": "[SEP]",
55
+ "strip_accents": null,
56
+ "tokenize_chinese_chars": true,
57
+ "tokenizer_class": "BertTokenizer",
58
+ "unk_token": "[UNK]"
59
+ }
onnx/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db4d615a10a12cfe134388d43d8fba48e909e7f7afa3e4d3b1bd2c24ff7cea3d
3
+ size 1334453533
rust_model.ot ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e7064a7b5b931482c99d04fb99f394254f2d3fd5f5d420f5d145b7d22f3cd56
3
+ size 1334478187
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}
tf_model.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:febce5a7915f2c894cdeb81cbe53bcc06ff3a8c22a96eaf9605aea0bbab172ba
3
+ size 1334871740
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": false, "max_len": 512}
vocab.txt ADDED
The diff for this file is too large to render. See raw diff