Hugging Face's logo

akdeniz27
/
bert-base-turkish-cased-ner-lora

Token Classification
PEFT
Turkish
Model card Files Community
1
bert-base-turkish-cased-ner-lora / README.md
akdeniz27's picture
akdeniz27
Update README.md
1f56ac4
preview code
|
raw
history blame
1.38 kB
metadata
library_name: peft
language: tr
widget:
  - text: Mustafa Kemal Atatürk 19 Mayıs 1919'da Samsun'a çıktı.

Training procedure

This is a fine-tuned model of base model "dbmdz/bert-base-turkish-cased" using the Parameter Efficient Fine Tuning (PEFT) with Low-Rank Adaptation (LoRA) technique using a reviewed version of well known Turkish NER dataset (https://github.com/stefan-it/turkish-bert/files/4558187/nerdata.txt).

Fine-tuning parameters: 

task = "ner"
model_checkpoint = "dbmdz/bert-base-turkish-cased"
batch_size = 16 
label_list = ['O', 'B-PER', 'I-PER', 'B-ORG', 'I-ORG', 'B-LOC', 'I-LOC']
max_length = 512 
learning_rate = 1e-3 
num_train_epochs = 10 
weight_decay = 0.01

PEFT Parameters 

inference_mode=False
r=16
lora_alpha=16
lora_dropout=0.1
bias="all"

How to use: 

model = AutoModelForTokenClassification.from_pretrained("akdeniz27/bert-base-turkish-cased-ner")
tokenizer = AutoTokenizer.from_pretrained("akdeniz27/bert-base-turkish-cased-ner")
ner = pipeline('ner', model=model, tokenizer=tokenizer, aggregation_strategy="first")
ner("your text here")

Pls refer "https://huggingface.co/transformers/_modules/transformers/pipelines/token_classification.html" for entity grouping with aggregation_strategy parameter.

Reference test results:

  • accuracy: 0.993104
  • f1: 0.948413
  • precision: 0.939789
  • recall: 0.957196