51la5 commited on
Commit
b4a90b3
1 Parent(s): 9731481

Upload 8 files

Browse files
README.md ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ datasets:
4
+ - conll2003
5
+ license: mit
6
+ ---
7
+ # bert-base-NER
8
+
9
+ ## Model description
10
+
11
+ **bert-large-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).
12
+
13
+ Specifically, this model is a *bert-large-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.
14
+
15
+ If you'd like to use a smaller BERT model fine-tuned on the same dataset, a [**bert-base-NER**](https://huggingface.co/dslim/bert-base-NER/) version is also available.
16
+
17
+
18
+ ## Intended uses & limitations
19
+
20
+ #### How to use
21
+
22
+ You can use this model with Transformers *pipeline* for NER.
23
+
24
+ ```python
25
+ from transformers import AutoTokenizer, AutoModelForTokenClassification
26
+ from transformers import pipeline
27
+
28
+ tokenizer = AutoTokenizer.from_pretrained("dslim/bert-base-NER")
29
+ model = AutoModelForTokenClassification.from_pretrained("dslim/bert-base-NER")
30
+
31
+ nlp = pipeline("ner", model=model, tokenizer=tokenizer)
32
+ example = "My name is Wolfgang and I live in Berlin"
33
+
34
+ ner_results = nlp(example)
35
+ print(ner_results)
36
+ ```
37
+
38
+ #### Limitations and bias
39
+
40
+ This model is limited by its training dataset of entity-annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains. Furthermore, the model occassionally tags subword tokens as entities and post-processing of results may be necessary to handle those cases.
41
+
42
+ ## Training data
43
+
44
+ This model was fine-tuned on English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.
45
+
46
+ The training dataset distinguishes between the beginning and continuation of an entity so that if there are back-to-back entities of the same type, the model can output where the second entity begins. As in the dataset, each token will be classified as one of the following classes:
47
+
48
+ Abbreviation|Description
49
+ -|-
50
+ O|Outside of a named entity
51
+ B-MIS |Beginning of a miscellaneous entity right after another miscellaneous entity
52
+ I-MIS | Miscellaneous entity
53
+ B-PER |Beginning of a person’s name right after another person’s name
54
+ I-PER |Person’s name
55
+ B-ORG |Beginning of an organization right after another organization
56
+ I-ORG |organization
57
+ B-LOC |Beginning of a location right after another location
58
+ I-LOC |Location
59
+
60
+
61
+ ### CoNLL-2003 English Dataset Statistics
62
+ This dataset was derived from the Reuters corpus which consists of Reuters news stories. You can read more about how this dataset was created in the CoNLL-2003 paper.
63
+ #### # of training examples per entity type
64
+ Dataset|LOC|MISC|ORG|PER
65
+ -|-|-|-|-
66
+ Train|7140|3438|6321|6600
67
+ Dev|1837|922|1341|1842
68
+ Test|1668|702|1661|1617
69
+ #### # of articles/sentences/tokens per dataset
70
+ Dataset |Articles |Sentences |Tokens
71
+ -|-|-|-
72
+ Train |946 |14,987 |203,621
73
+ Dev |216 |3,466 |51,362
74
+ Test |231 |3,684 |46,435
75
+
76
+ ## Training procedure
77
+
78
+ This model was trained on a single NVIDIA V100 GPU with recommended hyperparameters from the [original BERT paper](https://arxiv.org/pdf/1810.04805) which trained & evaluated the model on CoNLL-2003 NER task.
79
+
80
+ ## Eval results
81
+ metric|dev|test
82
+ -|-|-
83
+ f1 |95.7 |91.7
84
+ precision |95.3 |91.2
85
+ recall |96.1 |92.3
86
+
87
+ The test metrics are a little lower than the official Google BERT results which encoded document context & experimented with CRF. More on replicating the original results [here](https://github.com/google-research/bert/issues/223).
88
+
89
+ ### BibTeX entry and citation info
90
+
91
+ ```
92
+ @article{DBLP:journals/corr/abs-1810-04805,
93
+ author = {Jacob Devlin and
94
+ Ming{-}Wei Chang and
95
+ Kenton Lee and
96
+ Kristina Toutanova},
97
+ title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
98
+ Understanding},
99
+ journal = {CoRR},
100
+ volume = {abs/1810.04805},
101
+ year = {2018},
102
+ url = {http://arxiv.org/abs/1810.04805},
103
+ archivePrefix = {arXiv},
104
+ eprint = {1810.04805},
105
+ timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
106
+ biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
107
+ bibsource = {dblp computer science bibliography, https://dblp.org}
108
+ }
109
+ ```
110
+ ```
111
+ @inproceedings{tjong-kim-sang-de-meulder-2003-introduction,
112
+ title = "Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition",
113
+ author = "Tjong Kim Sang, Erik F. and
114
+ De Meulder, Fien",
115
+ booktitle = "Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003",
116
+ year = "2003",
117
+ url = "https://www.aclweb.org/anthology/W03-0419",
118
+ pages = "142--147",
119
+ }
120
+ ```
config.json ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "BertForTokenClassification"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.1,
6
+ "bos_token_id": null,
7
+ "directionality": "bidi",
8
+ "do_sample": false,
9
+ "eos_token_ids": null,
10
+ "finetuning_task": null,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.1,
13
+ "hidden_size": 1024,
14
+ "id2label": {
15
+ "0": "O",
16
+ "1": "B-MISC",
17
+ "2": "I-MISC",
18
+ "3": "B-PER",
19
+ "4": "I-PER",
20
+ "5": "B-ORG",
21
+ "6": "I-ORG",
22
+ "7": "B-LOC",
23
+ "8": "I-LOC"
24
+ },
25
+ "initializer_range": 0.02,
26
+ "intermediate_size": 4096,
27
+ "is_decoder": false,
28
+ "label2id": {
29
+ "B-LOC": 7,
30
+ "B-MISC": 1,
31
+ "B-ORG": 5,
32
+ "B-PER": 3,
33
+ "I-LOC": 8,
34
+ "I-MISC": 2,
35
+ "I-ORG": 6,
36
+ "I-PER": 4,
37
+ "O": 0
38
+ },
39
+ "layer_norm_eps": 1e-12,
40
+ "length_penalty": 1.0,
41
+ "max_length": 20,
42
+ "max_position_embeddings": 512,
43
+ "model_type": "bert",
44
+ "num_attention_heads": 16,
45
+ "num_beams": 1,
46
+ "num_hidden_layers": 24,
47
+ "num_labels": 9,
48
+ "num_return_sequences": 1,
49
+ "output_attentions": false,
50
+ "output_hidden_states": false,
51
+ "output_past": true,
52
+ "pad_token_id": 0,
53
+ "pooler_fc_size": 768,
54
+ "pooler_num_attention_heads": 12,
55
+ "pooler_num_fc_layers": 3,
56
+ "pooler_size_per_head": 128,
57
+ "pooler_type": "first_token_transform",
58
+ "pruned_heads": {},
59
+ "repetition_penalty": 1.0,
60
+ "temperature": 1.0,
61
+ "top_k": 50,
62
+ "top_p": 1.0,
63
+ "torchscript": false,
64
+ "type_vocab_size": 2,
65
+ "use_bfloat16": false,
66
+ "vocab_size": 28996
67
+ }
flax_model.msgpack ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d63d39520929bbeab16344157582be46f09a3ab43095b14f1b1586636a50f171
3
+ size 1330169528
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db4d615a10a12cfe134388d43d8fba48e909e7f7afa3e4d3b1bd2c24ff7cea3d
3
+ size 1334453533
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}
tf_model.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:febce5a7915f2c894cdeb81cbe53bcc06ff3a8c22a96eaf9605aea0bbab172ba
3
+ size 1334871740
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"do_lower_case": false, "max_len": 512}
vocab.txt ADDED
The diff for this file is too large to render. See raw diff