|
--- |
|
language: |
|
- en |
|
- ka |
|
license: mit |
|
tags: |
|
- flair |
|
- token-classification |
|
- sequence-tagger-model |
|
base_model: xlm-roberta-large |
|
widget: |
|
- text: ამით თავისი ქადაგება დაასრულა და დაბრუნდა იერუსალიმში . ერთ-ერთ გარე კედელზე |
|
არსებობს ერნესტო ჩე გევარას პორტრეტი . შაკოსკა“ ინახება ბრაზილიაში , სან-პაულუს |
|
ხელოვნების მუზეუმში . |
|
--- |
|
|
|
# Fine-tuned English-Georgian NER Model with Flair |
|
|
|
This Flair NER model was fine-tuned on the WikiANN dataset |
|
([Rahimi et al.](https://www.aclweb.org/anthology/P19-1015) splits) |
|
using XLM-R Large as backbone LM. |
|
|
|
**Notice**: The dataset is very problematic, because it was automatically constructed. |
|
|
|
We did manually inspect the development split of the Georgian data and found |
|
a lot of bad labeled examples, e.g. DVD ( 💿 ) as `ORG`. |
|
|
|
## Fine-Tuning |
|
|
|
The latest |
|
[Flair version](https://github.com/flairNLP/flair/tree/f30f5801df3f9e105ed078ec058b4e1152dd9159) |
|
is used for fine-tuning. |
|
|
|
We use English and Georgian training splits for fine-tuning and the |
|
development set of Georgian for evaluation. |
|
|
|
A hyper-parameter search over the following parameters with 5 different seeds per configuration is performed: |
|
|
|
* Batch Sizes: [`4`] |
|
* Learning Rates: [`5e-06`] |
|
|
|
More details can be found in this [repository](https://github.com/stefan-it/georgian-ner). |
|
|
|
## Results |
|
|
|
A hyper-parameter search with 5 different seeds per configuration is performed and micro F1-score on development set |
|
is reported: |
|
|
|
| Configuration | Seed 1 | Seed 2 | Seed 3 | Seed 4 | Seed 5 | Average | |
|
|-------------------|-------------|-------------|-------------|----------------|-------------|-----------------| |
|
| `bs4-e10-lr5e-06` | [0.9005][1] | [0.9012][2] | [0.9069][3] | [**0.905**][4] | [0.9048][5] | 0.9037 ± 0.0027 | |
|
|
|
[1]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-1 |
|
[2]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-2 |
|
[3]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-3 |
|
[4]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-4 |
|
[5]: https://hf.co/stefan-it/autotrain-flair-georgian-ner-xlm_r_large-bs4-e10-lr5e-06-5 |
|
|
|
The result in bold shows the performance of this model. |
|
|
|
Additionally, the Flair [training log](training.log) and [TensorBoard logs](tensorboard) are also uploaded to the model |
|
hub. |
