|
--- |
|
license: mit |
|
library_name: span-marker |
|
base_model: deepset/gelectra-large |
|
tags: |
|
- span-marker |
|
- token-classification |
|
- ner |
|
- named-entity-recognition |
|
pipeline_tag: token-classification |
|
widget: |
|
- text: "Jürgen Schmidhuber studierte ab 1983 Informatik und Mathematik an der TU München ." |
|
example_title: "Wikipedia" |
|
datasets: |
|
- gwlms/germeval2014 |
|
language: |
|
- de |
|
model-index: |
|
- name: SpanMarker with GELECTRA Large on GermEval 2014 NER Dataset by Stefan Schweter (@stefan-it) |
|
results: |
|
- task: |
|
type: token-classification |
|
name: Named Entity Recognition |
|
dataset: |
|
type: gwlms/germeval2014 |
|
name: GermEval 2014 |
|
split: test |
|
revision: f3647c56803ce67c08ee8d15f4611054c377b226 |
|
metrics: |
|
- type: f1 |
|
value: 0.8908 |
|
name: F1 |
|
- type: precision |
|
value: 0.8901 |
|
name: Precision |
|
- type: recall |
|
value: 0.8916 |
|
name: Recall |
|
metrics: |
|
- f1 |
|
- recall |
|
- precision |
|
--- |
|
|
|
# SpanMarker for GermEval 2014 NER |
|
|
|
This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model that |
|
was fine-tuned on the [GermEval 2014 NER Dataset](https://sites.google.com/site/germeval2014ner/home). |
|
|
|
The GermEval 2014 NER Shared Task builds on a new dataset with German Named Entity annotation with the following |
|
properties: The data was sampled from German Wikipedia and News Corpora as a collection of citations. The dataset |
|
covers over 31,000 sentences corresponding to over 590,000 tokens. The NER annotation uses the NoSta-D guidelines, |
|
which extend the Tübingen Treebank guidelines, using four main NER categories with sub-structure, and annotating |
|
embeddings among NEs such as `[ORG FC Kickers [LOC Darmstadt]]`. |
|
|
|
12 classes of Named Entites are annotated and must be recognized: four main classes `PER`son, `LOC`ation, `ORG`anisation, |
|
and `OTH`er and their subclasses by introducing two fine-grained labels: `-deriv` marks derivations from NEs such as |
|
"englisch" (“English”), and `-part` marks compounds including a NE as a subsequence deutschlandweit (“Germany-wide”). |
|
|
|
# Fine-Tuning |
|
|
|
We use the same hyper-parameters as used in the |
|
["German's Next Language Model"](https://aclanthology.org/2020.coling-main.598/) paper using the released |
|
[GELECTRA Large](https://huggingface.co/deepset/gelectra-large) model as backbone. |
|
|
|
Evaluation is performed with SpanMarkers internal evaluation code that uses `seqeval`. Additionally we use |
|
the official GermEval 2014 Evaluation Script for double-checking the results. A backup of the `nereval.py` script |
|
can be found [here](https://github.com/bplank/DaNplus/blob/master/scripts/nereval.perl). |
|
|
|
We fine-tune 5 models and upload the model with best F1-Score on development set. Results on development set are |
|
in brackets: |
|
|
|
| Model | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 | Avg. |
|
| ---------------------- | --------------- | --------------- | --------------- | --------------- | --------------- | --------------- |
|
| GELECTRA Large (5e-05) | (89.99) / 89.08 | (89.55) / 89.23 | (89.60) / 89.10 | (89.34) / 89.02 | (89.68) / 88.80 | (89.63) / 89.05 |
|
|
|
The best model achieves a final test score of 89.08%: |
|
|
|
```bash |
|
1. Strict, Combined Evaluation (official): |
|
Accuracy: 99.26%; |
|
Precision: 89.01%; |
|
Recall: 89.16%; |
|
FB1: 89.08 |
|
``` |
|
|
|
Scripts for [training](trainer.py) and [evaluation](evaluator.py) are also available. |
|
|
|
# Usage |
|
|
|
The fine-tuned model can be used like: |
|
|
|
```python |
|
from span_marker import SpanMarkerModel |
|
|
|
# Download from the 🤗 Hub |
|
model = SpanMarkerModel.from_pretrained("stefan-it/span-marker-gelectra-large-germeval14") |
|
|
|
# Run inference |
|
entities = model.predict("Jürgen Schmidhuber studierte ab 1983 Informatik und Mathematik an der TU München .") |
|
``` |
