gliner_ko / README.md
taeminlee's picture
Update README.md
fcaf888 verified
metadata
license: cc-by-nc-4.0
language:
  - ko
pipeline_tag: token-classification
library_name: gliner

Model Card for GLiNER-ko

GLiNER is a Named Entity Recognition (NER) model capable of identifying any entity type using a bidirectional transformer encoder (BERT-like). It provides a practical alternative to traditional NER models, which are limited to predefined entities, and Large Language Models (LLMs) that, despite their flexibility, are costly and large for resource-constrained scenarios.

This version has been trained on the various Korean NER dataset (Research purpose). Commercially permission versions are available (urchade/gliner_smallv2, urchade/gliner_mediumv2, urchade/gliner_largev2)

Links

Installation

To use this model, you must install the Korean fork of GLiNER Python library and mecab-ko:

!pip install gliner
!pip install python-mecab-ko

Usage

Once you've downloaded the GLiNER library, you can import the GLiNER class. You can then load this model using GLiNER.from_pretrained and predict entities with predict_entities.

from gliner import GLiNER

model = GLiNER.from_pretrained("taeminlee/gliner_ko")

text = """
ν”Όν„° 잭슨 κ²½(, 1961λ…„ 10μ›” 31일 ~ )은 λ‰΄μ§ˆλžœλ“œμ˜ μ˜ν™” 감독, 각본가, μ˜ν™” ν”„λ‘œλ“€μ„œμ΄λ‹€. J. R. R. ν†¨ν‚¨μ˜ μ†Œμ„€μ„ μ›μž‘μœΌλ‘œ ν•œ γ€Šλ°˜μ§€μ˜ μ œμ™• μ˜ν™” 3λΆ€μž‘γ€‹(2001λ…„~2003λ…„)의 κ°λ…μœΌλ‘œ κ°€μž₯ 유λͺ…ν•˜λ‹€. 2005λ…„μ—λŠ” 1933λ…„μž‘ ν‚Ήμ½©μ˜ λ¦¬λ©”μ΄ν¬μž‘ γ€Šν‚Ήμ½©(2005)γ€‹μ˜ 감독을 λ§‘μ•˜λ‹€.
"""

tta_labels = ["ARTIFACTS", "ANIMAL", "CIVILIZATION", "DATE", "EVENT", "STUDY_FIELD", "LOCATION", "MATERIAL", "ORGANIZATION", "PERSON", "PLANT", "QUANTITY", "TIME", "TERM", "THEORY"]

entities = model.predict_entities(text, labels)

for entity in entities:
    print(entity["text"], "=>", entity["label"])
ν”Όν„° 잭슨 κ²½ => PERSON
1961λ…„ 10μ›” 31일 ~ => DATE
λ‰΄μ§ˆλžœλ“œ => LOCATION
μ˜ν™” 감독 => CIVILIZATION
각본가 => CIVILIZATION
μ˜ν™” => CIVILIZATION
ν”„λ‘œλ“€μ„œ => CIVILIZATION
J. R. R. 톨킨 => PERSON
3λΆ€μž‘ => QUANTITY
2001λ…„~2003λ…„ => DATE
감독 => CIVILIZATION
2005λ…„ => DATE
1933λ…„μž‘ => DATE
킹콩 => ARTIFACTS
킹콩 => ARTIFACTS
2005 => DATE
감독 => CIVILIZATION

Named Entity Recognition benchmark result

Evaluate with the konne dev set

Model Precision (P) Recall (R) F1
Gliner-ko (t=0.5) 72.51% 79.82% 75.99%
Gliner Large-v2 (t=0.5) 34.33% 19.50% 24.87%
Gliner Multi (t=0.5) 40.94% 34.18% 37.26%
Pororo 70.25% 57.94% 63.50%

Model Authors

The model authors are:

Citation

@misc{zaratiana2023gliner,
      title={GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer}, 
      author={Urchade Zaratiana and Nadi Tomeh and Pierre Holat and Thierry Charnois},
      year={2023},
      eprint={2311.08526},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}