--- language: - en license: cc-by-sa-4.0 library_name: span-marker tags: - span-marker - token-classification - ner - named-entity-recognition - generated_from_span_marker_trainer datasets: - DFKI-SLT/few-nerd metrics: - precision - recall - f1 widget: - text: The Hebrew Union College libraries in Cincinnati and Los Angeles, the Library of Congress in Washington, D.C ., the Jewish Theological Seminary in New York City, and the Harvard University Library (which received donations of Deinard's texts from Lucius Nathan Littauer, housed in Widener and Houghton libraries) also have large collections of Deinard works. - text: Abu Abd Allah Muhammad al-Idrisi (1099–1165 or 1166), the Moroccan Muslim geographer, cartographer, Egyptologist and traveller who lived in Sicily at the court of King Roger II, mentioned this island, naming it جزيرة مليطمة ("jazīrat Malīṭma", "the island of Malitma ") on page 583 of his book "Nuzhat al-mushtaq fi ihtiraq ghal afaq", otherwise known as The Book of Roger, considered a geographic encyclopaedia of the medieval world. - text: The font is also used in the logo of the American rock band Greta Van Fleet, in the logo for Netflix show "Stranger Things ", and in the album art for rapper Logic's album "Supermarket ". - text: Caretaker manager George Goss led them on a run in the FA Cup, defeating Liverpool in round 4, to reach the semi-final at Stamford Bridge, where they were defeated 2–0 by Sheffield United on 28 March 1925. - text: In 1991, the National Science Foundation (NSF), which manages the U.S . Antarctic Program (US AP), honoured his memory by dedicating a state-of-the-art laboratory complex in his name, the Albert P. Crary Science and Engineering Center (CSEC) located in McMurdo Station. pipeline_tag: token-classification base_model: bert-base-cased model-index: - name: SpanMarker with bert-base-cased on DFKI-SLT/few-nerd results: - task: type: token-classification name: Named Entity Recognition dataset: name: Unknown type: DFKI-SLT/few-nerd split: test metrics: - type: f1 value: 0.7717265353418308 name: F1 - type: precision value: 0.7806212150810705 name: Precision - type: recall value: 0.7630322703838075 name: Recall --- # SpanMarker with bert-base-cased on DFKI-SLT/few-nerd This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [DFKI-SLT/few-nerd](https://huggingface.co/datasets/DFKI-SLT/few-nerd) dataset that can be used for Named Entity Recognition. This SpanMarker model uses [bert-base-cased](https://huggingface.co/bert-base-cased) as the underlying encoder. ## Model Details ### Model Description - **Model Type:** SpanMarker - **Encoder:** [bert-base-cased](https://huggingface.co/bert-base-cased) - **Maximum Sequence Length:** 256 tokens - **Maximum Entity Length:** 8 words - **Training Dataset:** [DFKI-SLT/few-nerd](https://huggingface.co/datasets/DFKI-SLT/few-nerd) - **Language:** en - **License:** cc-by-sa-4.0 ### Model Sources - **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER) - **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf) ### Model Labels | Label | Examples | |:-------------|:-------------------------------------------------------------------------------| | art | "Time", "The Seven Year Itch", "Imelda de ' Lambertazzi" | | building | "Boston Garden", "Sheremetyevo International Airport", "Henry Ford Museum" | | event | "French Revolution", "Iranian Constitutional Revolution", "Russian Revolution" | | location | "Croatian", "the Republic of Croatia", "Mediterranean Basin" | | organization | "IAEA", "Texas Chicken", "Church 's Chicken" | | other | "N-terminal lipid", "BAR", "Amphiphysin" | | person | "Hicks", "Edmund Payne", "Ellaline Terriss" | | product | "100EX", "Phantom", "Corvettes - GT1 C6R" | ## Evaluation ### Metrics | Label | Precision | Recall | F1 | |:-------------|:----------|:-------|:-------| | **all** | 0.7806 | 0.7630 | 0.7717 | | art | 0.7465 | 0.7395 | 0.7430 | | building | 0.6027 | 0.7184 | 0.6555 | | event | 0.6178 | 0.5438 | 0.5784 | | location | 0.8138 | 0.8547 | 0.8338 | | organization | 0.7359 | 0.6613 | 0.6966 | | other | 0.7397 | 0.6166 | 0.6726 | | person | 0.8845 | 0.9071 | 0.8957 | | product | 0.7056 | 0.5932 | 0.6446 | ## Uses ### Direct Use for Inference ```python from span_marker import SpanMarkerModel # Download from the 🤗 Hub model = SpanMarkerModel.from_pretrained("span_marker_model_id") # Run inference entities = model.predict("Caretaker manager George Goss led them on a run in the FA Cup, defeating Liverpool in round 4, to reach the semi-final at Stamford Bridge, where they were defeated 2–0 by Sheffield United on 28 March 1925.") ``` ### Downstream Use You can finetune this model on your own dataset.
Click to expand ```python from span_marker import SpanMarkerModel, Trainer # Download from the 🤗 Hub model = SpanMarkerModel.from_pretrained("span_marker_model_id") # Specify a Dataset with "tokens" and "ner_tag" columns dataset = load_dataset("conll2003") # For example CoNLL2003 # Initialize a Trainer using the pretrained model & dataset trainer = Trainer( model=model, train_dataset=dataset["train"], eval_dataset=dataset["validation"], ) trainer.train() trainer.save_model("span_marker_model_id-finetuned") ```
## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:----------------------|:----|:--------|:----| | Sentence length | 1 | 24.4956 | 163 | | Entities per sentence | 0 | 2.5439 | 35 | ### Training Hyperparameters - learning_rate: 5e-05 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - gradient_accumulation_steps: 2 - total_train_batch_size: 8 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 1 ### Training Results | Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy | |:------:|:----:|:---------------:|:--------------------:|:-----------------:|:-------------:|:-------------------:| | 0.1629 | 200 | 0.0359 | 0.6908 | 0.6298 | 0.6589 | 0.9053 | | 0.3259 | 400 | 0.0237 | 0.7535 | 0.7018 | 0.7267 | 0.9227 | | 0.4888 | 600 | 0.0216 | 0.7659 | 0.7438 | 0.7547 | 0.9333 | | 0.6517 | 800 | 0.0208 | 0.7730 | 0.7550 | 0.7639 | 0.9344 | | 0.8147 | 1000 | 0.0197 | 0.7805 | 0.7567 | 0.7684 | 0.9372 | | 0.9776 | 1200 | 0.0194 | 0.7771 | 0.7634 | 0.7702 | 0.9381 | ### Framework Versions - Python: 3.10.12 - SpanMarker: 1.4.0 - Transformers: 4.34.0 - PyTorch: 2.0.1+cu118 - Datasets: 2.14.5 - Tokenizers: 0.14.1 ## Citation ### BibTeX ``` @software{Aarsen_SpanMarker, author = {Aarsen, Tom}, license = {Apache-2.0}, title = {{SpanMarker for Named Entity Recognition}}, url = {https://github.com/tomaarsen/SpanMarkerNER} } ```