zhang19991111 commited on
Commit
4157dbe
1 Parent(s): 18490fd

Upload 9 files

Browse files
README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: cc-by-sa-4.0
4
+ library_name: span-marker
5
+ tags:
6
+ - span-marker
7
+ - token-classification
8
+ - ner
9
+ - named-entity-recognition
10
+ - generated_from_span_marker_trainer
11
+ metrics:
12
+ - precision
13
+ - recall
14
+ - f1
15
+ widget:
16
+ - text: Altitude measurements based on near - IR imaging in H and Hcont filters showed
17
+ that the deeper BS2 clouds were located near the methane condensation level (
18
+ ≈1.2bars ) , while BS1 was generally ∼500 mb above that level ( at lower pressures
19
+ ) .
20
+ - text: However , our model predicts different performance for large enough memory
21
+ - access latency and validates the intuition that the dynamic programming algorithm
22
+ performs better on these machines .
23
+ - text: We established a P fertilizer need map based on integrating results from the
24
+ two systems .
25
+ - text: Here , we have addressed this limitation for the endodermal lineage by developing
26
+ a defined culture system to expand and differentiate human foregut stem cells
27
+ ( hFSCs ) derived from hPSCs . hFSCs can self - renew while maintaining their
28
+ capacity to differentiate into pancreatic and hepatic cells .
29
+ - text: The accumulated percentage gain from selection amounted to 51%/1 % lower Striga
30
+ infestation ( measured by area under Striga number progress curve , ASNPC ) ,
31
+ 46%/62 % lower downy mildew incidence , and 49%/31 % higher panicle yield of the
32
+ C5 - FS compared to the mean of the genepool parents at Sadoré / Cinzana , respectively
33
+ .
34
+ pipeline_tag: token-classification
35
+ base_model: allenai/specter2_base
36
+ model-index:
37
+ - name: SpanMarker with allenai/specter2_base on my-data
38
+ results:
39
+ - task:
40
+ type: token-classification
41
+ name: Named Entity Recognition
42
+ dataset:
43
+ name: my-data
44
+ type: unknown
45
+ split: test
46
+ metrics:
47
+ - type: f1
48
+ value: 0.6906354515050167
49
+ name: F1
50
+ - type: precision
51
+ value: 0.7108433734939759
52
+ name: Precision
53
+ - type: recall
54
+ value: 0.6715447154471544
55
+ name: Recall
56
+ ---
57
+
58
+ # SpanMarker with allenai/specter2_base on my-data
59
+
60
+ This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model that can be used for Named Entity Recognition. This SpanMarker model uses [allenai/specter2_base](https://huggingface.co/allenai/specter2_base) as the underlying encoder.
61
+
62
+ ## Model Details
63
+
64
+ ### Model Description
65
+ - **Model Type:** SpanMarker
66
+ - **Encoder:** [allenai/specter2_base](https://huggingface.co/allenai/specter2_base)
67
+ - **Maximum Sequence Length:** 256 tokens
68
+ - **Maximum Entity Length:** 8 words
69
+ <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
70
+ - **Language:** en
71
+ - **License:** cc-by-sa-4.0
72
+
73
+ ### Model Sources
74
+
75
+ - **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER)
76
+ - **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf)
77
+
78
+ ### Model Labels
79
+ | Label | Examples |
80
+ |:---------|:--------------------------------------------------------------------------------------------------------|
81
+ | Data | "Depth time - series", "defect", "an overall mitochondrial" |
82
+ | Material | "cross - shore measurement locations", "the subject 's fibroblasts", "COXI , COXII and COXIII subunits" |
83
+ | Method | "an approximation", "EFSA", "in vitro" |
84
+ | Process | "intake", "a significant reduction of synthesis", "translation" |
85
+
86
+ ## Evaluation
87
+
88
+ ### Metrics
89
+ | Label | Precision | Recall | F1 |
90
+ |:---------|:----------|:-------|:-------|
91
+ | **all** | 0.7108 | 0.6715 | 0.6906 |
92
+ | Data | 0.6591 | 0.6138 | 0.6356 |
93
+ | Material | 0.795 | 0.7910 | 0.7930 |
94
+ | Method | 0.5 | 0.45 | 0.4737 |
95
+ | Process | 0.6898 | 0.6293 | 0.6582 |
96
+
97
+ ## Uses
98
+
99
+ ### Direct Use for Inference
100
+
101
+ ```python
102
+ from span_marker import SpanMarkerModel
103
+
104
+ # Download from the 🤗 Hub
105
+ model = SpanMarkerModel.from_pretrained("span-marker-allenai/specter2_base-me")
106
+ # Run inference
107
+ entities = model.predict("We established a P fertilizer need map based on integrating results from the two systems .")
108
+ ```
109
+
110
+ ### Downstream Use
111
+ You can finetune this model on your own dataset.
112
+
113
+ <details><summary>Click to expand</summary>
114
+
115
+ ```python
116
+ from span_marker import SpanMarkerModel, Trainer
117
+
118
+ # Download from the 🤗 Hub
119
+ model = SpanMarkerModel.from_pretrained("span-marker-allenai/specter2_base-me")
120
+
121
+ # Specify a Dataset with "tokens" and "ner_tag" columns
122
+ dataset = load_dataset("conll2003") # For example CoNLL2003
123
+
124
+ # Initialize a Trainer using the pretrained model & dataset
125
+ trainer = Trainer(
126
+ model=model,
127
+ train_dataset=dataset["train"],
128
+ eval_dataset=dataset["validation"],
129
+ )
130
+ trainer.train()
131
+ trainer.save_model("span-marker-allenai/specter2_base-me-finetuned")
132
+ ```
133
+ </details>
134
+
135
+ <!--
136
+ ### Out-of-Scope Use
137
+
138
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
139
+ -->
140
+
141
+ <!--
142
+ ## Bias, Risks and Limitations
143
+
144
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
145
+ -->
146
+
147
+ <!--
148
+ ### Recommendations
149
+
150
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
151
+ -->
152
+
153
+ ## Training Details
154
+
155
+ ### Training Set Metrics
156
+ | Training set | Min | Median | Max |
157
+ |:----------------------|:----|:--------|:----|
158
+ | Sentence length | 3 | 25.6049 | 106 |
159
+ | Entities per sentence | 0 | 5.2439 | 22 |
160
+
161
+ ### Training Hyperparameters
162
+ - learning_rate: 5e-05
163
+ - train_batch_size: 8
164
+ - eval_batch_size: 8
165
+ - seed: 42
166
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
167
+ - lr_scheduler_type: linear
168
+ - lr_scheduler_warmup_ratio: 0.1
169
+ - num_epochs: 10
170
+
171
+ ### Framework Versions
172
+ - Python: 3.10.12
173
+ - SpanMarker: 1.5.0
174
+ - Transformers: 4.36.2
175
+ - PyTorch: 2.0.1+cu118
176
+ - Datasets: 2.16.1
177
+ - Tokenizers: 0.15.0
178
+
179
+ ## Citation
180
+
181
+ ### BibTeX
182
+ ```
183
+ @software{Aarsen_SpanMarker,
184
+ author = {Aarsen, Tom},
185
+ license = {Apache-2.0},
186
+ title = {{SpanMarker for Named Entity Recognition}},
187
+ url = {https://github.com/tomaarsen/SpanMarkerNER}
188
+ }
189
+ ```
190
+
191
+ <!--
192
+ ## Glossary
193
+
194
+ *Clearly define terms in order to be accessible across audiences.*
195
+ -->
196
+
197
+ <!--
198
+ ## Model Card Authors
199
+
200
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
201
+ -->
202
+
203
+ <!--
204
+ ## Model Card Contact
205
+
206
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
207
+ -->
added_tokens.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "<end>": 31091,
3
+ "<start>": 31090
4
+ }
config.json ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "SpanMarkerModel"
4
+ ],
5
+ "encoder": {
6
+ "_name_or_path": "allenai/specter2_base",
7
+ "adapters": {
8
+ "adapters": {},
9
+ "config_map": {},
10
+ "fusion_config_map": {},
11
+ "fusions": {}
12
+ },
13
+ "add_cross_attention": false,
14
+ "architectures": [
15
+ "BertModel"
16
+ ],
17
+ "attention_probs_dropout_prob": 0.1,
18
+ "bad_words_ids": null,
19
+ "begin_suppress_tokens": null,
20
+ "bos_token_id": null,
21
+ "chunk_size_feed_forward": 0,
22
+ "classifier_dropout": null,
23
+ "cross_attention_hidden_size": null,
24
+ "decoder_start_token_id": null,
25
+ "diversity_penalty": 0.0,
26
+ "do_sample": false,
27
+ "early_stopping": false,
28
+ "encoder_no_repeat_ngram_size": 0,
29
+ "eos_token_id": null,
30
+ "exponential_decay_length_penalty": null,
31
+ "finetuning_task": null,
32
+ "forced_bos_token_id": null,
33
+ "forced_eos_token_id": null,
34
+ "hidden_act": "gelu",
35
+ "hidden_dropout_prob": 0.1,
36
+ "hidden_size": 768,
37
+ "id2label": {
38
+ "0": "O",
39
+ "1": "Data",
40
+ "2": "Material",
41
+ "3": "Method",
42
+ "4": "Process"
43
+ },
44
+ "initializer_range": 0.02,
45
+ "intermediate_size": 3072,
46
+ "is_decoder": false,
47
+ "is_encoder_decoder": false,
48
+ "label2id": {
49
+ "Data": 1,
50
+ "Material": 2,
51
+ "Method": 3,
52
+ "O": 0,
53
+ "Process": 4
54
+ },
55
+ "layer_norm_eps": 1e-12,
56
+ "length_penalty": 1.0,
57
+ "max_length": 20,
58
+ "max_position_embeddings": 512,
59
+ "min_length": 0,
60
+ "model_type": "bert",
61
+ "no_repeat_ngram_size": 0,
62
+ "num_attention_heads": 12,
63
+ "num_beam_groups": 1,
64
+ "num_beams": 1,
65
+ "num_hidden_layers": 12,
66
+ "num_return_sequences": 1,
67
+ "output_attentions": false,
68
+ "output_hidden_states": false,
69
+ "output_scores": false,
70
+ "pad_token_id": 0,
71
+ "position_embedding_type": "absolute",
72
+ "prefix": null,
73
+ "problem_type": null,
74
+ "pruned_heads": {},
75
+ "remove_invalid_values": false,
76
+ "repetition_penalty": 1.0,
77
+ "return_dict": true,
78
+ "return_dict_in_generate": false,
79
+ "sep_token_id": null,
80
+ "suppress_tokens": null,
81
+ "task_specific_params": null,
82
+ "temperature": 1.0,
83
+ "tf_legacy_loss": false,
84
+ "tie_encoder_decoder": false,
85
+ "tie_word_embeddings": true,
86
+ "tokenizer_class": null,
87
+ "top_k": 50,
88
+ "top_p": 1.0,
89
+ "torch_dtype": "float32",
90
+ "torchscript": false,
91
+ "transformers_version": "4.36.2",
92
+ "type_vocab_size": 2,
93
+ "typical_p": 1.0,
94
+ "use_bfloat16": false,
95
+ "use_cache": true,
96
+ "vocab_size": 31096
97
+ },
98
+ "entity_max_length": 8,
99
+ "marker_max_length": 128,
100
+ "max_next_context": null,
101
+ "max_prev_context": null,
102
+ "model_max_length": 256,
103
+ "model_max_length_default": 512,
104
+ "model_type": "span-marker",
105
+ "span_marker_version": "1.5.0",
106
+ "torch_dtype": "float32",
107
+ "trained_with_document_context": false,
108
+ "transformers_version": "4.36.2",
109
+ "vocab_size": 31096
110
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e0c789490988f2ce89f0dcd35e0b268b6e9696ae97ead87da0132205272843d
3
+ size 439747140
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": true,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "[PAD]",
6
+ "lstrip": false,
7
+ "normalized": false,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "101": {
13
+ "content": "[UNK]",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "102": {
21
+ "content": "[CLS]",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "103": {
29
+ "content": "[SEP]",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "104": {
37
+ "content": "[MASK]",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "31090": {
45
+ "content": "<start>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "31091": {
53
+ "content": "<end>",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ }
60
+ },
61
+ "clean_up_tokenization_spaces": true,
62
+ "cls_token": "[CLS]",
63
+ "do_basic_tokenize": true,
64
+ "do_lower_case": true,
65
+ "entity_max_length": 8,
66
+ "marker_max_length": 128,
67
+ "mask_token": "[MASK]",
68
+ "model_max_length": 256,
69
+ "never_split": null,
70
+ "pad_token": "[PAD]",
71
+ "sep_token": "[SEP]",
72
+ "strip_accents": null,
73
+ "tokenize_chinese_chars": true,
74
+ "tokenizer_class": "BertTokenizer",
75
+ "unk_token": "[UNK]"
76
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3402aba2424bc17fea8b4d9e1f63674ae9cb4d510b83ff618718c43c2571700f
3
+ size 4283
vocab.txt ADDED
The diff for this file is too large to render. See raw diff