tomaarsen HF staff commited on
Commit
67a38be
1 Parent(s): 4d06631

Upload model

Browse files
README.md ADDED
@@ -0,0 +1,236 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - tl
4
+ license: gpl-3.0
5
+ library_name: span-marker
6
+ tags:
7
+ - span-marker
8
+ - token-classification
9
+ - ner
10
+ - named-entity-recognition
11
+ - generated_from_span_marker_trainer
12
+ datasets:
13
+ - ljvmiranda921/tlunified-ner
14
+ metrics:
15
+ - precision
16
+ - recall
17
+ - f1
18
+ widget:
19
+ - text: MANILA - Binalewala ng Philippine National Police (PNP) nitong Sabado ang
20
+ posibleng paglulunsad ng tinatawag na " sympathy attacks " ng Moro National Liberation
21
+ Front (MNLF) at Abu Sayyaf matapos arestuhin si Indanan, Sulu Mayor Alvarez Isnaji.
22
+ - text: Pinatawan din ng apat na buwang suspensyon si Herma Gonzales - Escudero, chief
23
+ revenue officer III ng BIR - Cotabato City, dahil sa kasong dishonesty at limang
24
+ kaso ng perjury sa Municipal Trial Court ng Cotabato City . Bunga ito ng kanyang
25
+ kabiguan na ideklara sa kanyang SALN noong 2002 - 2004 ang 200 metro kwadradong
26
+ lote sa South Cotabato at Toyota Revo noong 2001 SALN at undervaluation ng kanyang
27
+ mga ari - arian sa lalawigan noong 2000 - 2004 SALN.
28
+ - text: Sa tila pagpapabaya sa mga magsasaka, sinabi ni Escudero na hindi mangyayari
29
+ ang pangarap ng Department of Agriculture (DA) na maging self - sufficient ang
30
+ Pilipinas sa bigas.
31
+ - text: MANILA - Tiniyak ng pinuno ng Government Service Insurance System (GSIS) na
32
+ tatapatan nito ang pro - Meralco advertisement ni Judy Ann Santos upang isulong
33
+ ang kanyang posisyon na dapat ibaba ang singil sa kuryente.
34
+ - text: Idinagdag ni South Cotabato Rep Darlene Antonino - Custodio, na illegal na
35
+ ipagpaliban ang halalan sa ARMM kung ang gagamitin lamang basehan ay ang ipapasang
36
+ panukala ng Kongreso.
37
+ pipeline_tag: token-classification
38
+ co2_eq_emissions:
39
+ emissions: 22.090476722294312
40
+ source: codecarbon
41
+ training_type: fine-tuning
42
+ on_cloud: false
43
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
44
+ ram_total_size: 31.777088165283203
45
+ hours_used: 0.238
46
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
47
+ base_model: bert-base-multilingual-cased
48
+ model-index:
49
+ - name: SpanMarker with bert-base-multilingual-cased on TLUnified
50
+ results:
51
+ - task:
52
+ type: token-classification
53
+ name: Named Entity Recognition
54
+ dataset:
55
+ name: TLUnified
56
+ type: ljvmiranda921/tlunified-ner
57
+ split: test
58
+ metrics:
59
+ - type: f1
60
+ value: 0.8886810102899907
61
+ name: F1
62
+ - type: precision
63
+ value: 0.8736971183323115
64
+ name: Precision
65
+ - type: recall
66
+ value: 0.9041878172588832
67
+ name: Recall
68
+ ---
69
+
70
+ # SpanMarker with bert-base-multilingual-cased on TLUnified
71
+
72
+ This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [TLUnified](https://huggingface.co/datasets/ljvmiranda921/tlunified-ner) dataset that can be used for Named Entity Recognition. This SpanMarker model uses [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) as the underlying encoder.
73
+
74
+ ## Model Details
75
+
76
+ ### Model Description
77
+ - **Model Type:** SpanMarker
78
+ - **Encoder:** [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased)
79
+ - **Maximum Sequence Length:** 256 tokens
80
+ - **Maximum Entity Length:** 8 words
81
+ - **Training Dataset:** [TLUnified](https://huggingface.co/datasets/ljvmiranda921/tlunified-ner)
82
+ - **Language:** tl
83
+ - **License:** gpl-3.0
84
+
85
+ ### Model Sources
86
+
87
+ - **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER)
88
+ - **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf)
89
+
90
+ ### Model Labels
91
+ | Label | Examples |
92
+ |:------|:----------------------------------------------------------------------------------------------------|
93
+ | LOC | "Israel", "Batasan", "United States" |
94
+ | ORG | "MMDA", "International Monitoring Team", "Coordinating Committees for the Cessation of Hostilities" |
95
+ | PER | "Puno", "Fernando", "Villavicencio" |
96
+
97
+ ## Evaluation
98
+
99
+ ### Metrics
100
+ | Label | Precision | Recall | F1 |
101
+ |:--------|:----------|:-------|:-------|
102
+ | **all** | 0.8737 | 0.9042 | 0.8887 |
103
+ | LOC | 0.8830 | 0.9084 | 0.8955 |
104
+ | ORG | 0.7579 | 0.8587 | 0.8052 |
105
+ | PER | 0.9264 | 0.9220 | 0.9242 |
106
+
107
+ ## Uses
108
+
109
+ ### Direct Use for Inference
110
+
111
+ ```python
112
+ from span_marker import SpanMarkerModel
113
+
114
+ # Download from the 🤗 Hub
115
+ model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-mbert-base-tlunified")
116
+ # Run inference
117
+ entities = model.predict("Idinagdag ni South Cotabato Rep Darlene Antonino - Custodio, na illegal na ipagpaliban ang halalan sa ARMM kung ang gagamitin lamang basehan ay ang ipapasang panukala ng Kongreso.")
118
+ ```
119
+
120
+ ### Downstream Use
121
+ You can finetune this model on your own dataset.
122
+
123
+ <details><summary>Click to expand</summary>
124
+
125
+ ```python
126
+ from span_marker import SpanMarkerModel, Trainer
127
+
128
+ # Download from the 🤗 Hub
129
+ model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-mbert-base-tlunified")
130
+
131
+ # Specify a Dataset with "tokens" and "ner_tag" columns
132
+ dataset = load_dataset("conll2003") # For example CoNLL2003
133
+
134
+ # Initialize a Trainer using the pretrained model & dataset
135
+ trainer = Trainer(
136
+ model=model,
137
+ train_dataset=dataset["train"],
138
+ eval_dataset=dataset["validation"],
139
+ )
140
+ trainer.train()
141
+ trainer.save_model("tomaarsen/span-marker-mbert-base-tlunified-finetuned")
142
+ ```
143
+ </details>
144
+
145
+ <!--
146
+ ### Out-of-Scope Use
147
+
148
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
149
+ -->
150
+
151
+ <!--
152
+ ## Bias, Risks and Limitations
153
+
154
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
155
+ -->
156
+
157
+ <!--
158
+ ### Recommendations
159
+
160
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
161
+ -->
162
+
163
+ ## Training Details
164
+
165
+ ### Training Set Metrics
166
+ | Training set | Min | Median | Max |
167
+ |:----------------------|:----|:--------|:----|
168
+ | Sentence length | 1 | 31.7625 | 150 |
169
+ | Entities per sentence | 0 | 2.0661 | 38 |
170
+
171
+ ### Training Hyperparameters
172
+ - learning_rate: 5e-05
173
+ - train_batch_size: 16
174
+ - eval_batch_size: 16
175
+ - seed: 42
176
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
177
+ - lr_scheduler_type: linear
178
+ - lr_scheduler_warmup_ratio: 0.1
179
+ - num_epochs: 3
180
+
181
+ ### Training Results
182
+ | Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy |
183
+ |:------:|:----:|:---------------:|:--------------------:|:-----------------:|:-------------:|:-------------------:|
184
+ | 0.6803 | 400 | 0.0074 | 0.8552 | 0.8835 | 0.8691 | 0.9774 |
185
+ | 1.3605 | 800 | 0.0072 | 0.8709 | 0.9034 | 0.8869 | 0.9798 |
186
+ | 2.0408 | 1200 | 0.0070 | 0.8753 | 0.9053 | 0.8900 | 0.9812 |
187
+ | 2.7211 | 1600 | 0.0065 | 0.8876 | 0.9003 | 0.8939 | 0.9807 |
188
+
189
+ ### Environmental Impact
190
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
191
+ - **Carbon Emitted**: 0.022 kg of CO2
192
+ - **Hours Used**: 0.238 hours
193
+
194
+ ### Training Hardware
195
+ - **On Cloud**: No
196
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
197
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
198
+ - **RAM Size**: 31.78 GB
199
+
200
+ ### Framework Versions
201
+ - Python: 3.9.16
202
+ - SpanMarker: 1.5.1.dev
203
+ - Transformers: 4.30.0
204
+ - PyTorch: 2.0.1+cu118
205
+ - Datasets: 2.14.0
206
+ - Tokenizers: 0.13.3
207
+
208
+ ## Citation
209
+
210
+ ### BibTeX
211
+ ```
212
+ @software{Aarsen_SpanMarker,
213
+ author = {Aarsen, Tom},
214
+ license = {Apache-2.0},
215
+ title = {{SpanMarker for Named Entity Recognition}},
216
+ url = {https://github.com/tomaarsen/SpanMarkerNER}
217
+ }
218
+ ```
219
+
220
+ <!--
221
+ ## Glossary
222
+
223
+ *Clearly define terms in order to be accessible across audiences.*
224
+ -->
225
+
226
+ <!--
227
+ ## Model Card Authors
228
+
229
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
230
+ -->
231
+
232
+ <!--
233
+ ## Model Card Contact
234
+
235
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
236
+ -->
added_tokens.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "<end>": 119548,
3
+ "<start>": 119547
4
+ }
config.json ADDED
@@ -0,0 +1,135 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "SpanMarkerModel"
4
+ ],
5
+ "encoder": {
6
+ "_name_or_path": "bert-base-multilingual-cased",
7
+ "add_cross_attention": false,
8
+ "architectures": [
9
+ "BertForMaskedLM"
10
+ ],
11
+ "attention_probs_dropout_prob": 0.1,
12
+ "bad_words_ids": null,
13
+ "begin_suppress_tokens": null,
14
+ "bos_token_id": null,
15
+ "chunk_size_feed_forward": 0,
16
+ "classifier_dropout": null,
17
+ "cross_attention_hidden_size": null,
18
+ "decoder_start_token_id": null,
19
+ "directionality": "bidi",
20
+ "diversity_penalty": 0.0,
21
+ "do_sample": false,
22
+ "early_stopping": false,
23
+ "encoder_no_repeat_ngram_size": 0,
24
+ "eos_token_id": null,
25
+ "exponential_decay_length_penalty": null,
26
+ "finetuning_task": null,
27
+ "forced_bos_token_id": null,
28
+ "forced_eos_token_id": null,
29
+ "hidden_act": "gelu",
30
+ "hidden_dropout_prob": 0.1,
31
+ "hidden_size": 768,
32
+ "id2label": {
33
+ "0": "O",
34
+ "1": "B-PER",
35
+ "2": "I-PER",
36
+ "3": "B-ORG",
37
+ "4": "I-ORG",
38
+ "5": "B-LOC",
39
+ "6": "I-LOC"
40
+ },
41
+ "initializer_range": 0.02,
42
+ "intermediate_size": 3072,
43
+ "is_decoder": false,
44
+ "is_encoder_decoder": false,
45
+ "label2id": {
46
+ "B-LOC": 5,
47
+ "B-ORG": 3,
48
+ "B-PER": 1,
49
+ "I-LOC": 6,
50
+ "I-ORG": 4,
51
+ "I-PER": 2,
52
+ "O": 0
53
+ },
54
+ "layer_norm_eps": 1e-12,
55
+ "length_penalty": 1.0,
56
+ "max_length": 20,
57
+ "max_position_embeddings": 512,
58
+ "min_length": 0,
59
+ "model_type": "bert",
60
+ "no_repeat_ngram_size": 0,
61
+ "num_attention_heads": 12,
62
+ "num_beam_groups": 1,
63
+ "num_beams": 1,
64
+ "num_hidden_layers": 12,
65
+ "num_return_sequences": 1,
66
+ "output_attentions": false,
67
+ "output_hidden_states": false,
68
+ "output_scores": false,
69
+ "pad_token_id": 0,
70
+ "pooler_fc_size": 768,
71
+ "pooler_num_attention_heads": 12,
72
+ "pooler_num_fc_layers": 3,
73
+ "pooler_size_per_head": 128,
74
+ "pooler_type": "first_token_transform",
75
+ "position_embedding_type": "absolute",
76
+ "prefix": null,
77
+ "problem_type": null,
78
+ "pruned_heads": {},
79
+ "remove_invalid_values": false,
80
+ "repetition_penalty": 1.0,
81
+ "return_dict": true,
82
+ "return_dict_in_generate": false,
83
+ "sep_token_id": null,
84
+ "suppress_tokens": null,
85
+ "task_specific_params": null,
86
+ "temperature": 1.0,
87
+ "tf_legacy_loss": false,
88
+ "tie_encoder_decoder": false,
89
+ "tie_word_embeddings": true,
90
+ "tokenizer_class": null,
91
+ "top_k": 50,
92
+ "top_p": 1.0,
93
+ "torch_dtype": null,
94
+ "torchscript": false,
95
+ "transformers_version": "4.30.0",
96
+ "type_vocab_size": 2,
97
+ "typical_p": 1.0,
98
+ "use_bfloat16": false,
99
+ "use_cache": true,
100
+ "vocab_size": 119549
101
+ },
102
+ "entity_max_length": 8,
103
+ "id2label": {
104
+ "0": "O",
105
+ "1": "LOC",
106
+ "2": "ORG",
107
+ "3": "PER"
108
+ },
109
+ "id2reduced_id": {
110
+ "0": 0,
111
+ "1": 3,
112
+ "2": 3,
113
+ "3": 2,
114
+ "4": 2,
115
+ "5": 1,
116
+ "6": 1
117
+ },
118
+ "label2id": {
119
+ "LOC": 1,
120
+ "O": 0,
121
+ "ORG": 2,
122
+ "PER": 3
123
+ },
124
+ "marker_max_length": 128,
125
+ "max_next_context": null,
126
+ "max_prev_context": null,
127
+ "model_max_length": 256,
128
+ "model_max_length_default": 512,
129
+ "model_type": "span-marker",
130
+ "span_marker_version": "1.5.1.dev",
131
+ "torch_dtype": "float32",
132
+ "trained_with_document_context": false,
133
+ "transformers_version": "4.30.0",
134
+ "vocab_size": 119549
135
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:434c3a0d8fb796b8cf20aafa33d9226b1a7e0f953a2c3f1b76f0b2d006c4d33d
3
+ size 711517877
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": true,
3
+ "clean_up_tokenization_spaces": true,
4
+ "cls_token": "[CLS]",
5
+ "do_lower_case": false,
6
+ "entity_max_length": 8,
7
+ "marker_max_length": 128,
8
+ "mask_token": "[MASK]",
9
+ "model_max_length": 256,
10
+ "pad_token": "[PAD]",
11
+ "sep_token": "[SEP]",
12
+ "strip_accents": null,
13
+ "tokenize_chinese_chars": true,
14
+ "tokenizer_class": "BertTokenizer",
15
+ "unk_token": "[UNK]"
16
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff