asarymsakova/bert-base-gl-ner

Named Entity Recognition BERT Model for Galician

This repository includes a fine-tuned model for Named Entity Recognition (NER) annotation for Galician. It is a fine-tuned version of marcosgg/bert-base-gl-cased trained for token classification with the BIO tagging scheme.

The model recognises four entity types:

Tag	Entity type
PER	Person
LOC	Location
ORG	Organisation
MISC	Miscellaneous

Model description

• Architecture: BertForTokenClassification (BERT base, 12 layers, hidden size 768, 12 attention heads)
• Base model: marcosgg/bert-base-gl-cased
• Language: Galician (gl)
• Task: Named Entity Recognition (token classification)
• Labels (BIO): B-PER, I-PER, B-LOC, I-LOC, B-ORG, I-ORG, B-MISC, I-MISC, O
• Cased: yes
• Maximum sequence length: 512 tokens

Intended use

The model is intended for automatic Named Entity Recognition in Galician text, tagging tokens as person (PER), location (LOC), organisation (ORG) or miscellaneous (MISC) entities.

Training and evaluation data

The training, development and test data are available at:

https://github.com/albinasarymsakova/Named-Entity-Recognition-Resources-for-Galician

The data is distributed in JSON Lines format, with parallel lists of tokens and BIO labels per sentence.

Evaluation results

Test sets

Results (%) on the five individual Galician NER test sets, using the BIO scheme and entity-level (seqeval) scoring:

Test set	F1	Precision	Recall
SLI NERC test	88.81	87.76	89.89
TreeGal test	86.38	86.03	86.73
PUD	83.82	83.45	84.19
CorNER	91.51	91.46	91.56
LREC	84.60	83.29	85.96
Mixed (concatenation of all test sets)	86.30
Average over the five test sets	87.03

How to use

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

model_id = "asarymsakova/bert-base-gl-ner"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

ner = pipeline(
    "token-classification",
    model=model,
    tokenizer=tokenizer,
    aggregation_strategy="simple",
)

text = "A Universidade de Santiago de Compostela está situada en Galicia."
for entity in ner(text):
    print(entity["entity_group"], entity["word"], round(entity["score"], 3))

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 64
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1

License

This resource is made available under the terms of the GNU Affero General Public License v3.0 (AGPL-3.0). See https://www.gnu.org/licenses/agpl-3.0.html for the full license text. The resource is distributed without any warranty.

Information and contact

• Albina Sarymsakova — albina.sarymsakova.es@gmail.com
• Marcos Garcia — marcos.garcia.gonzalez@usc.gal

Citation

If you use this model, please cite this repository and the accompanying NER resources for Galician: https://github.com/albinasarymsakova/Named-Entity-Recognition-Resources-for-Galician