Package: daluke
DaLUKE: The Entity-aware, Danish Language ModelImplementation of the knowledge-enhanced transformerLUKEpretrained on the Danish Wikipedia and evaluated on named entity recognition (NER).Installationpip install dalukeFor including optional requirements that are necessary for training and general analysis:pip install daluke[full]Python 3.8 or newer is required.ExplanationFor an explanation of the model, see ourbachelor's thesisor the originalLUKE paper.UsageInference on simple NER or masked language modeling (MLM) examplesPythonFor performing NER predictionsfromdalukeimportAutoNERDaLUKE,predict_nerdaluke=AutoNERDaLUKE()document="Det Kgl. Bibliotek forvalter Danmarks største tekstsamling, der strækker sig fra middelalderen til det nyeste litteratur."iob_list=predict_ner(document,daluke)For testing MLM predictionsfromdalukeimportAutoMLMDaLUKE,predict_mlmdaluke=AutoMLMDaLUKE()# Empty list => No entity annotations in the stringdocument="Professor i astrofysik, [MASK] [MASK], udtaler til avisen, at den nye måling sandsynligvis ikke er en fejl."best_prediction,table=predict_mlm(document,list(),daluke)CLIdalukener--text"Thomas Delaney fører Danmark til sejr ved EM i fodbold."dalukemasked--text"Slutresultatet af kampen mellem Danmark og Rusland bliver [MASK]-[MASK]."For Windows, or systems where#!/usr/bin/env python3is not linked to the correct Python interpreter, the commandpython -m daluke.api.clican be used instead ofdaluke.Training DaLUKE yourselfThis part shows how to recreate the entire DaLUKE training pipeline from dataset preparation to fine-tuning.
This guide is designed to be run in a bash shell.
If you use Windows, you will probably have to make some modifications to the shell scripts used.# Download forked luke submodulegitsubmoduleupdate--init--recursive# Install requirementspipinstall-rrequirements.txt
pipinstall-roptional-requirements.txt
pipinstall-rluke/requirements.txt# Build dataset# The script performs all the steps of building the dataset, including downloading the Danish Wikipedia# You only need to modify DATA_PATH to where you want the data to be saved# Be aware that this takes several hoursdev/build_data.sh# Start pretraining using default hyperparameterspythondaluke/pretrain/run.py<INSERTDATA_PATHHERE>-cconfigs/pretrain-main.ini--name$NAME--save-every5--epochs150--namedaluke--fp16# Optional: Make plots of pretrainingpythondaluke/plot/plot_pretraining.py<DATA_PATH>/daluke# Fine-tune on DaNEpythondaluke/collect_modelfile.py<DATA_PATH>/daluke<DATA_PATH>/ner/daluke.tar.gz
pythondaluke/ner/run.py<DATA_PATH>/ner/daluke-cconfigs/main-finetune.ini--model<DATA_PATH>/ner/daluke.tar.gz--namefinetune--eval# Evaluate on DaNE test setpythondaluke/ner/run_eval.py<DATA_PATH>/ner/daluke/finetune--model<DATA_PATH>/ner/daluke/finetune/daluke_ner_best.tar.gz# Optional: Fine-tuning plotspythondaluke/plot/plot_finetune_ner.py<DATA_PATH>/ner/daluke/finetune/train-resultsHistory0.0.5- Added batching in Python API NER forward passing0.0.4- Added a Python API for maintaining a stateful model and performing CWR, MLM and NER predictions0.0.3: Finalization of Bachelor's Project- Allowed specifying entity spans in masked word prediction CLI0.0.2- CLI made working on Windows0.0.1- Simple single-example CLI released