--- library_name: span-marker tags: - span-marker - token-classification - ner - named-entity-recognition - generated_from_span_marker_trainer datasets: - DFKI-SLT/few-nerd metrics: - precision - recall - f1 widget: - text: In response, in May or June 1125, a 3,000-strong Crusader coalition commanded by King Baldwin II of Jerusalem confronted and defeated the 15,000-strong Muslim coalition at the Battle of Azaz, raising the siege of the town. - text: Cardenal made several visits to Jesuit universities in the United States, including the University of Detroit Mercy in 2013, and the John Carroll University in 2014. - text: Other super-spreaders, defined as those that transmit SARS to at least eight other people, included the incidents at the Hotel Metropole in Hong Kong, the Amoy Gardens apartment complex in Hong Kong and one in an acute care hospital in Toronto, Ontario, Canada. - text: The District Court for the Northern District of California rejected 321 Studios' claims for declaratory relief, holding that both DVD Copy Plus and DVD-X Copy violated the DMCA and that the DMCA was not unconstitutional. - text: The Sunday Edition is a television programme broadcast on the ITV Network in the United Kingdom focusing on political interview and discussion, produced by ITV Productions. pipeline_tag: token-classification model-index: - name: SpanMarker results: - task: type: token-classification name: Named Entity Recognition dataset: name: Unknown type: DFKI-SLT/few-nerd split: test metrics: - type: f1 value: 0.703084859534267 name: F1 - type: precision value: 0.7034273336857051 name: Precision - type: recall value: 0.7027427186979075 name: Recall --- # SpanMarker This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [DFKI-SLT/few-nerd](https://huggingface.co/datasets/DFKI-SLT/few-nerd) dataset that can be used for Named Entity Recognition. ## Model Details ### Model Description - **Model Type:** SpanMarker - **Maximum Sequence Length:** 256 tokens - **Maximum Entity Length:** 8 words - **Training Dataset:** [DFKI-SLT/few-nerd](https://huggingface.co/datasets/DFKI-SLT/few-nerd) ### Model Sources - **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER) - **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf) ### Model Labels | Label | Examples | |:-----------------------------------------|:---------------------------------------------------------------------------------------------------------| | art-broadcastprogram | "Street Cents", "Corazones", "The Gale Storm Show : Oh , Susanna" | | art-film | "L'Atlantide", "Shawshank Redemption", "Bosch" | | art-music | "Champion Lover", "Atkinson , Danko and Ford ( with Brockie and Hilton )", "Hollywood Studio Symphony" | | art-other | "Aphrodite of Milos", "The Today Show", "Venus de Milo" | | art-painting | "Production/Reproduction", "Cofiwch Dryweryn", "Touit" | | art-writtenart | "Time", "Imelda de ' Lambertazzi", "The Seven Year Itch" | | building-airport | "Sheremetyevo International Airport", "Luton Airport", "Newark Liberty International Airport" | | building-hospital | "Yeungnam University Hospital", "Memorial Sloan-Kettering Cancer Center", "Hokkaido University Hospital" | | building-hotel | "Radisson Blu Sea Plaza Hotel", "Flamingo Hotel", "The Standard Hotel" | | building-library | "British Library", "Berlin State Library", "Bayerische Staatsbibliothek" | | building-other | "Communiplex", "Henry Ford Museum", "Alpha Recording Studios" | | building-restaurant | "Carnegie Deli", "Trumbull", "Fatburger" | | building-sportsfacility | "Sports Center", "Boston Garden", "Glenn Warner Soccer Facility" | | building-theater | "Sanders Theatre", "Pittsburgh Civic Light Opera", "National Paris Opera" | | event-attack/battle/war/militaryconflict | "Vietnam War", "Jurist", "Easter Offensive" | | event-disaster | "1990s North Korean famine", "the 1912 North Mount Lyell Disaster", "1693 Sicily earthquake" | | event-election | "1982 Mitcham and Morden by-election", "Elections to the European Parliament", "March 1898 elections" | | event-other | "Eastwood Scoring Stage", "Union for a Popular Movement", "Masaryk Democratic Movement" | | event-protest | "French Revolution", "Iranian Constitutional Revolution", "Russian Revolution" | | event-sportsevent | "World Cup", "National Champions", "Stanley Cup" | | location-GPE | "Mediterranean Basin", "the Republic of Croatia", "Croatian" | | location-bodiesofwater | "Arthur Kill", "Atatürk Dam Lake", "Norfolk coast" | | location-island | "Staten Island", "new Samsat district", "Laccadives" | | location-mountain | "Miteirya Ridge", "Ruweisat Ridge", "Salamander Glacier" | | location-other | "Northern City Line", "Victoria line", "Cartuther" | | location-park | "Painted Desert Community Complex Historic District", "Gramercy Park", "Shenandoah National Park" | | location-road/railway/highway/transit | "NJT", "Newark-Elizabeth Rail Link", "Friern Barnet Road" | | organization-company | "Church 's Chicken", "Texas Chicken", "Dixy Chicken" | | organization-education | "Barnard College", "MIT", "Belfast Royal Academy and the Ulster College of Physical Education" | | organization-government/governmentagency | "Diet", "Supreme Court", "Congregazione dei Nobili" | | organization-media/newspaper | "Al Jazeera", "Clash", "TimeOut Melbourne" | | organization-other | "Defence Sector C", "4th Army", "IAEA" | | organization-politicalparty | "Al Wafa ' Islamic", "Shimpotō", "Kenseitō" | | organization-religion | "Jewish", "UPCUSA", "Christian" | | organization-showorganization | "Mr. Mister", "Lizzy", "Bochumer Symphoniker" | | organization-sportsleague | "NHL", "First Division", "China League One" | | organization-sportsteam | "Arsenal", "Luc Alphand Aventures", "Tottenham" | | other-astronomything | "Algol", "Zodiac", "`` Caput Larvae ''" | | other-award | "Order of the Republic of Guinea and Nigeria", "GCON", "Grand Commander of the Order of the Niger" | | other-biologything | "Amphiphysin", "BAR", "N-terminal lipid" | | other-chemicalthing | "sulfur", "uranium", "carbon dioxide" | | other-currency | "$", "Travancore Rupee", "lac crore" | | other-disease | "hypothyroidism", "bladder cancer", "French Dysentery Epidemic of 1779" | | other-educationaldegree | "BSc ( Hons ) in physics", "Master", "Bachelor" | | other-god | "El", "Raijin", "Fujin" | | other-language | "Latin", "English", "Breton-speaking" | | other-law | "United States Freedom Support Act", "Thirty Years ' Peace", "Leahy–Smith America Invents Act ( AIA" | | other-livingthing | "insects", "monkeys", "patchouli" | | other-medical | "pediatrician", "Pediatrics", "amitriptyline" | | person-actor | "Edmund Payne", "Tchéky Karyo", "Ellaline Terriss" | | person-artist/author | "Gaetano Donizett", "George Axelrod", "Hicks" | | person-athlete | "Tozawa", "Jaguar", "Neville" | | person-director | "Bob Swaim", "Frank Darabont", "Richard Quine" | | person-other | "Holden", "Richard Benson", "Campbell" | | person-politician | "Rivière", "Emeric", "William" | | person-scholar | "Stalmine", "Wurdack", "Stedman" | | person-soldier | "Krukenberg", "Joachim Ziegler", "Helmuth Weidling" | | product-airplane | "EC135T2 CPDS", "Spey-equipped FGR.2s", "Luton" | | product-car | "100EX", "Corvettes - GT1 C6R", "Phantom" | | product-food | "yakiniku", "V. labrusca", "red grape" | | product-game | "Airforce Delta", "Splinter Cell", "Hardcore RPG" | | product-other | "X11", "Fairbottom Bobs", "PDP-1" | | product-ship | "Essex", "HMS `` Chinkara ''", "Congress" | | product-software | "Wikipedia", "Apdf", "AmiPDF" | | product-train | "High Speed Trains", "Royal Scots Grey", "55022" | | product-weapon | "ZU-23-2M Wróbel", "AR-15 's", "ZU-23-2MR Wróbel II" | ## Evaluation ### Metrics | Label | Precision | Recall | F1 | |:-----------------------------------------|:----------|:-------|:-------| | **all** | 0.7034 | 0.7027 | 0.7031 | | art-broadcastprogram | 0.6024 | 0.5904 | 0.5963 | | art-film | 0.7761 | 0.7533 | 0.7645 | | art-music | 0.7825 | 0.7551 | 0.7685 | | art-other | 0.4193 | 0.3327 | 0.3710 | | art-painting | 0.5882 | 0.5263 | 0.5556 | | art-writtenart | 0.6819 | 0.6488 | 0.6649 | | building-airport | 0.8064 | 0.8352 | 0.8205 | | building-hospital | 0.7282 | 0.8022 | 0.7634 | | building-hotel | 0.7033 | 0.7245 | 0.7138 | | building-library | 0.7550 | 0.7380 | 0.7464 | | building-other | 0.5867 | 0.5840 | 0.5853 | | building-restaurant | 0.6205 | 0.5216 | 0.5667 | | building-sportsfacility | 0.6113 | 0.7976 | 0.6921 | | building-theater | 0.7060 | 0.7495 | 0.7271 | | event-attack/battle/war/militaryconflict | 0.7945 | 0.7395 | 0.7660 | | event-disaster | 0.5604 | 0.5604 | 0.5604 | | event-election | 0.4286 | 0.1484 | 0.2204 | | event-other | 0.4885 | 0.4400 | 0.4629 | | event-protest | 0.3798 | 0.4759 | 0.4225 | | event-sportsevent | 0.6198 | 0.6162 | 0.6180 | | location-GPE | 0.8157 | 0.8552 | 0.8350 | | location-bodiesofwater | 0.7268 | 0.7690 | 0.7473 | | location-island | 0.7504 | 0.6842 | 0.7158 | | location-mountain | 0.7352 | 0.7298 | 0.7325 | | location-other | 0.4427 | 0.3104 | 0.3649 | | location-park | 0.7153 | 0.6856 | 0.7001 | | location-road/railway/highway/transit | 0.7090 | 0.7324 | 0.7205 | | organization-company | 0.6963 | 0.7061 | 0.7012 | | organization-education | 0.7994 | 0.7986 | 0.7990 | | organization-government/governmentagency | 0.5524 | 0.4533 | 0.4980 | | organization-media/newspaper | 0.6513 | 0.6656 | 0.6584 | | organization-other | 0.5978 | 0.5375 | 0.5661 | | organization-politicalparty | 0.6793 | 0.7315 | 0.7044 | | organization-religion | 0.5575 | 0.6131 | 0.5840 | | organization-showorganization | 0.6035 | 0.5839 | 0.5935 | | organization-sportsleague | 0.6393 | 0.6610 | 0.6499 | | organization-sportsteam | 0.7259 | 0.7796 | 0.7518 | | other-astronomything | 0.7794 | 0.8024 | 0.7907 | | other-award | 0.7180 | 0.6649 | 0.6904 | | other-biologything | 0.6864 | 0.6238 | 0.6536 | | other-chemicalthing | 0.5688 | 0.6036 | 0.5856 | | other-currency | 0.6996 | 0.8423 | 0.7643 | | other-disease | 0.6591 | 0.7410 | 0.6977 | | other-educationaldegree | 0.6114 | 0.6198 | 0.6156 | | other-god | 0.6486 | 0.7181 | 0.6816 | | other-language | 0.6507 | 0.8313 | 0.7300 | | other-law | 0.6934 | 0.7331 | 0.7127 | | other-livingthing | 0.6019 | 0.6605 | 0.6298 | | other-medical | 0.5124 | 0.5214 | 0.5169 | | person-actor | 0.8384 | 0.8051 | 0.8214 | | person-artist/author | 0.7122 | 0.7531 | 0.7321 | | person-athlete | 0.8318 | 0.8422 | 0.8370 | | person-director | 0.7083 | 0.7365 | 0.7221 | | person-other | 0.6833 | 0.6737 | 0.6785 | | person-politician | 0.6807 | 0.6836 | 0.6822 | | person-scholar | 0.5397 | 0.5209 | 0.5301 | | person-soldier | 0.5053 | 0.5920 | 0.5452 | | product-airplane | 0.6617 | 0.6692 | 0.6654 | | product-car | 0.7313 | 0.7132 | 0.7222 | | product-food | 0.5787 | 0.5787 | 0.5787 | | product-game | 0.7364 | 0.7140 | 0.7250 | | product-other | 0.5567 | 0.4210 | 0.4795 | | product-ship | 0.6842 | 0.6842 | 0.6842 | | product-software | 0.6495 | 0.6648 | 0.6570 | | product-train | 0.5942 | 0.5924 | 0.5933 | | product-weapon | 0.6435 | 0.5353 | 0.5844 | ## Uses ### Direct Use for Inference ```python from span_marker import SpanMarkerModel # Download from the 🤗 Hub model = SpanMarkerModel.from_pretrained("supreethrao/instructNER_fewnerd_xl") # Run inference entities = model.predict("The Sunday Edition is a television programme broadcast on the ITV Network in the United Kingdom focusing on political interview and discussion, produced by ITV Productions.") ``` ### Downstream Use You can finetune this model on your own dataset.
Click to expand ```python from span_marker import SpanMarkerModel, Trainer # Download from the 🤗 Hub model = SpanMarkerModel.from_pretrained("supreethrao/instructNER_fewnerd_xl") # Specify a Dataset with "tokens" and "ner_tag" columns dataset = load_dataset("conll2003") # For example CoNLL2003 # Initialize a Trainer using the pretrained model & dataset trainer = Trainer( model=model, train_dataset=dataset["train"], eval_dataset=dataset["validation"], ) trainer.train() trainer.save_model("supreethrao/instructNER_fewnerd_xl-finetuned") ```
## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:----------------------|:----|:--------|:----| | Sentence length | 1 | 24.4945 | 267 | | Entities per sentence | 0 | 2.5832 | 88 | ### Training Hyperparameters - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - distributed_type: multi-GPU - num_devices: 2 - total_train_batch_size: 32 - total_eval_batch_size: 32 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 3 - mixed_precision_training: Native AMP ### Framework Versions - Python: 3.10.13 - SpanMarker: 1.5.0 - Transformers: 4.35.2 - PyTorch: 2.1.1 - Datasets: 2.15.0 - Tokenizers: 0.15.0 ## Citation ### BibTeX ``` @software{Aarsen_SpanMarker, author = {Aarsen, Tom}, license = {Apache-2.0}, title = {{SpanMarker for Named Entity Recognition}}, url = {https://github.com/tomaarsen/SpanMarkerNER} } ```