Example code does not work with new versions of transformers
Example should be updated to this to work with current transformers-library (also torch needs to be installed in addition to transformers):
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
model_checkpoint = "Kansallisarkisto/finbert-ner"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
model = AutoModelForTokenClassification.from_pretrained(model_checkpoint)
token_classifier = pipeline(
"token-classification",
model=model,
tokenizer=tokenizer,
aggregation_strategy="simple",
)
predictions = token_classifier("Helsingistä tuli Suomen suuriruhtinaskunnan pääkaupunki vuonna 1812.")
print(predictions)
Hi!
Thanks for notifying us on this problem. I tested the old code example with Transformers version 5.6.2, and it seems to be still working without errors. However I updated the example code in the repo to use AutoTokenizer and AutoModelForTokenClassification, as this also allows more customized use cases.
Best regards,
National Archives AI team