|
--- |
|
language: |
|
- tl |
|
license: gpl-3.0 |
|
library_name: span-marker |
|
tags: |
|
- span-marker |
|
- token-classification |
|
- ner |
|
- named-entity-recognition |
|
- generated_from_span_marker_trainer |
|
datasets: |
|
- ljvmiranda921/tlunified-ner |
|
metrics: |
|
- precision |
|
- recall |
|
- f1 |
|
widget: |
|
- text: MANILA - Binalewala ng Philippine National Police (PNP) nitong Sabado ang |
|
posibleng paglulunsad ng tinatawag na " sympathy attacks " ng Moro National Liberation |
|
Front (MNLF) at Abu Sayyaf matapos arestuhin si Indanan, Sulu Mayor Alvarez Isnaji. |
|
- text: Pinatawan din ng apat na buwang suspensyon si Herma Gonzales - Escudero, chief |
|
revenue officer III ng BIR - Cotabato City, dahil sa kasong dishonesty at limang |
|
kaso ng perjury sa Municipal Trial Court ng Cotabato City . Bunga ito ng kanyang |
|
kabiguan na ideklara sa kanyang SALN noong 2002 - 2004 ang 200 metro kwadradong |
|
lote sa South Cotabato at Toyota Revo noong 2001 SALN at undervaluation ng kanyang |
|
mga ari - arian sa lalawigan noong 2000 - 2004 SALN. |
|
- text: Sa tila pagpapabaya sa mga magsasaka, sinabi ni Escudero na hindi mangyayari |
|
ang pangarap ng Department of Agriculture (DA) na maging self - sufficient ang |
|
Pilipinas sa bigas. |
|
- text: MANILA - Tiniyak ng pinuno ng Government Service Insurance System (GSIS) na |
|
tatapatan nito ang pro - Meralco advertisement ni Judy Ann Santos upang isulong |
|
ang kanyang posisyon na dapat ibaba ang singil sa kuryente. |
|
- text: Idinagdag ni South Cotabato Rep Darlene Antonino - Custodio, na illegal na |
|
ipagpaliban ang halalan sa ARMM kung ang gagamitin lamang basehan ay ang ipapasang |
|
panukala ng Kongreso. |
|
pipeline_tag: token-classification |
|
co2_eq_emissions: |
|
emissions: 22.090476722294312 |
|
source: codecarbon |
|
training_type: fine-tuning |
|
on_cloud: false |
|
cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K |
|
ram_total_size: 31.777088165283203 |
|
hours_used: 0.238 |
|
hardware_used: 1 x NVIDIA GeForce RTX 3090 |
|
base_model: bert-base-multilingual-cased |
|
model-index: |
|
- name: SpanMarker with bert-base-multilingual-cased on TLUnified |
|
results: |
|
- task: |
|
type: token-classification |
|
name: Named Entity Recognition |
|
dataset: |
|
name: TLUnified |
|
type: ljvmiranda921/tlunified-ner |
|
split: test |
|
metrics: |
|
- type: f1 |
|
value: 0.8886810102899907 |
|
name: F1 |
|
- type: precision |
|
value: 0.8736971183323115 |
|
name: Precision |
|
- type: recall |
|
value: 0.9041878172588832 |
|
name: Recall |
|
--- |
|
|
|
# SpanMarker with bert-base-multilingual-cased on TLUnified |
|
|
|
This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [TLUnified](https://huggingface.co/datasets/ljvmiranda921/tlunified-ner) dataset that can be used for Named Entity Recognition. This SpanMarker model uses [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) as the underlying encoder. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** SpanMarker |
|
- **Encoder:** [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) |
|
- **Maximum Sequence Length:** 256 tokens |
|
- **Maximum Entity Length:** 8 words |
|
- **Training Dataset:** [TLUnified](https://huggingface.co/datasets/ljvmiranda921/tlunified-ner) |
|
- **Language:** tl |
|
- **License:** gpl-3.0 |
|
|
|
### Model Sources |
|
|
|
- **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER) |
|
- **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf) |
|
|
|
### Model Labels |
|
| Label | Examples | |
|
|:------|:----------------------------------------------------------------------------------------------------| |
|
| LOC | "Israel", "Batasan", "United States" | |
|
| ORG | "MMDA", "International Monitoring Team", "Coordinating Committees for the Cessation of Hostilities" | |
|
| PER | "Puno", "Fernando", "Villavicencio" | |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
| Label | Precision | Recall | F1 | |
|
|:--------|:----------|:-------|:-------| |
|
| **all** | 0.8737 | 0.9042 | 0.8887 | |
|
| LOC | 0.8830 | 0.9084 | 0.8955 | |
|
| ORG | 0.7579 | 0.8587 | 0.8052 | |
|
| PER | 0.9264 | 0.9220 | 0.9242 | |
|
|
|
## Uses |
|
|
|
### Direct Use for Inference |
|
|
|
```python |
|
from span_marker import SpanMarkerModel |
|
|
|
# Download from the 🤗 Hub |
|
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-mbert-base-tlunified") |
|
# Run inference |
|
entities = model.predict("Idinagdag ni South Cotabato Rep Darlene Antonino - Custodio, na illegal na ipagpaliban ang halalan sa ARMM kung ang gagamitin lamang basehan ay ang ipapasang panukala ng Kongreso.") |
|
``` |
|
|
|
### Downstream Use |
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
```python |
|
from span_marker import SpanMarkerModel, Trainer |
|
|
|
# Download from the 🤗 Hub |
|
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-mbert-base-tlunified") |
|
|
|
# Specify a Dataset with "tokens" and "ner_tag" columns |
|
dataset = load_dataset("conll2003") # For example CoNLL2003 |
|
|
|
# Initialize a Trainer using the pretrained model & dataset |
|
trainer = Trainer( |
|
model=model, |
|
train_dataset=dataset["train"], |
|
eval_dataset=dataset["validation"], |
|
) |
|
trainer.train() |
|
trainer.save_model("tomaarsen/span-marker-mbert-base-tlunified-finetuned") |
|
``` |
|
</details> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Set Metrics |
|
| Training set | Min | Median | Max | |
|
|:----------------------|:----|:--------|:----| |
|
| Sentence length | 1 | 31.7625 | 150 | |
|
| Entities per sentence | 0 | 2.0661 | 38 | |
|
|
|
### Training Hyperparameters |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_ratio: 0.1 |
|
- num_epochs: 3 |
|
|
|
### Training Results |
|
| Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy | |
|
|:------:|:----:|:---------------:|:--------------------:|:-----------------:|:-------------:|:-------------------:| |
|
| 0.6803 | 400 | 0.0074 | 0.8552 | 0.8835 | 0.8691 | 0.9774 | |
|
| 1.3605 | 800 | 0.0072 | 0.8709 | 0.9034 | 0.8869 | 0.9798 | |
|
| 2.0408 | 1200 | 0.0070 | 0.8753 | 0.9053 | 0.8900 | 0.9812 | |
|
| 2.7211 | 1600 | 0.0065 | 0.8876 | 0.9003 | 0.8939 | 0.9807 | |
|
|
|
### Environmental Impact |
|
Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon). |
|
- **Carbon Emitted**: 0.022 kg of CO2 |
|
- **Hours Used**: 0.238 hours |
|
|
|
### Training Hardware |
|
- **On Cloud**: No |
|
- **GPU Model**: 1 x NVIDIA GeForce RTX 3090 |
|
- **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K |
|
- **RAM Size**: 31.78 GB |
|
|
|
### Framework Versions |
|
- Python: 3.9.16 |
|
- SpanMarker: 1.5.1.dev |
|
- Transformers: 4.30.0 |
|
- PyTorch: 2.0.1+cu118 |
|
- Datasets: 2.14.0 |
|
- Tokenizers: 0.13.3 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
``` |
|
@software{Aarsen_SpanMarker, |
|
author = {Aarsen, Tom}, |
|
license = {Apache-2.0}, |
|
title = {{SpanMarker for Named Entity Recognition}}, |
|
url = {https://github.com/tomaarsen/SpanMarkerNER} |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |