YurtsAI/ner-document-context

SpanMarker with roberta-base on YurtsAI/named_entity_recognition_document_context

This is a SpanMarker model trained on the YurtsAI/named_entity_recognition_document_context dataset that can be used for Named Entity Recognition. This SpanMarker model uses roberta-base as the underlying encoder.

Model Details

Model Description

• Model Type: SpanMarker
• Encoder: roberta-base
• Maximum Sequence Length: 256 tokens
• Maximum Entity Length: 11 words
• Training Dataset: YurtsAI/named_entity_recognition_document_context
• Language: en

Model Sources

• Repository: SpanMarker on GitHub
• Thesis: SpanMarker For Named Entity Recognition

Model Labels

Label	Examples
art-broadcastprogram	"television program", "Origin of the Gods", "reality show"
art-film	"a video of a successful grant proposal", "'The Matrix '", "film crew"
art-music	"a new album by Beyoncé", "Yesterday by The Beatles", "favorite music CD"
art-other	"art therapy", "play", "Mona Lisa"
art-painting	"vibrant street art scene", "through art", "painting"
art-writtenart	"'The Lost Gods '", "Book 1", "environmental science book"
building-airport	"airport", "major airport", "an airport"
building-hospital	"New York hospital", "local hospital", "hospital"
building-hotel	"hotel", "new hotel in Austin", "a giant hotel"
building-library	"new library", "library", "new , state-of-the-art library"
building-other	"10-story building", "headquarters building", "factory building"
building-restaurant	"new restaurant", "our upscale restaurant", "restaurant"
building-sportsfacility	"sports facility", "Union Park Sports Complex", "city 's sports center"
building-theater	"the local theater", "theater in downtown", "theater"
datetime-absolute	"January 10 , 2020", "January 17 , 2025 at 14:00", "March 25th"
datetime-authored	"2023-02-22", "2019-04-15", "2020-02-15"
datetime-range	"2010-2015", "Q4 2019", "Friday to Sunday"
datetime-relative	"next week 's appointment", "last Saturday", "next week"
event-attack/battle/war/militaryconflict	"attacks/wars", "The", "A"
event-disaster	"My", "To", "disaster"
event-election	"the election for the mayor", "upcoming election", "election season"
event-other	"conference", "annual 4th of july BBQ", "charity gala"
event-protest	"protest", "protest last saturday", "protest rally"
event-sportsevent	"sports event", "annual tennis tournament", "biggest sports event of the year"
location-bodiesofwater	"ocean", "Lake Como", "Lake Michigan"
location-gpe	"Italy", "Texas", "city"
location-island	"Island Radio", "Caribbean island", "island"
location-mountain	"mountain terrain", "the mountain", "mountain"
location-other	"low-lying areas of the city", "advertising hub", "backyard"
location-park	"park", "location-park", "the park"
location-road/railway/highway/transit	"Greyhound network", "road", "train journey"
organization-company	"local company", "Verizon", "a company"
organization-education	"Harvard University", "UW", "University of Arizona"
organization-government/governmentagency	"Red Cross", "local government", "SEC"
organization-media/newspaper	"The New York Times", "media organizations", "Army Times"
organization-other	"Cognizant", "Better World Foundation", "conservation organization"
organization-politicalparty	"Spaceship of Progress Party", "Libertarian Party", "Green Party"
organization-religion	"local church", "the power of prayer", "diamatists"
organization-showorganization	"Royal Shakespeare Company", "Earth 's Edge Theater Company", "Cosmic Theater group"
organization-sportsleague	"International Swimming Federation", "NBA league", "NFL"
organization-sportsteam	"soccer team", "Syracuse Orange football team", "Seattle Seahawks"
other-astronomything	"latest discoveries in the field of astronomy", "Galactic Conference Best Recipe Award-winning recipe book", "astronomy camp"
other-award	"other-award", "annual tech show awards", "Nobel Peace Prize"
other-biologything	"salmon 's gene for cold adaptation", "terrain", "the forces that drive you"
other-chemicalthing	"Overall", "The", "In"
other-currency	"US dollars", "Japanese Yen", "$ 500,000"
other-disease	"malaria", "type 1 diabetes", "the common cold"
other-educationaldegree	"master 's degree", "thesis", "Ph.D in food science"
other-god	"Peter Pan", "divine", "Zeus the god"
other-language	"English", "Amharic", "Sanskrit"
other-law	"legislation", "professorial separation laws", "Clean Air Act"
other-livingthing	"We", "To", "flowers"
other-medical	"antibiotics", "medical treatment", "necessary testing protocols"
person-actor	"Emma Stone", "Dr. Steven Spielberg", "Jennifer Lawrence"
person-artist/author	"Chuck Close", "artist 's new album", "Jane Smith"
person-athlete	"athlete friend", "LeBron James", "John and Sally"
person-director	"John Oliver", "favorite director", "Dr. Johnson"
person-other	"your", "HR representative", "therapist or counselor"
person-politician	"To", "At", "Secretary of State"
person-scholar	"Dr. John Smith", "Dr. Johnson", "a scholar of comparative religion"
person-soldier	"veterans", "the brave soldiers", "a soldier"
product-airplane	"Cessna 172", "company 's fleet of private airplanes", "airline"
product-car	"leased car", "your car", "car"
product-food	"StarBites", "food truck business", "ice cream"
product-game	"the 'Train to Nowhere ' game", "board game", "screen protector"
product-other	"new medicine", "acting software", "table"
product-ship	"research ship", "ship", "a ship"
product-software	"software", "instruction manual", "pizza ordering app"
product-train	"Universal Sonicator", "train", "the train"
product-weapon	"Flip Flops", "Sno Blaster", "SecurityFirst"

Evaluation

Metrics

Label	Precision	Recall	F1
all	0.6189	0.2850	0.3903
art-broadcastprogram	0.0	0.0	0.0
art-film	0.0	0.0	0.0
art-music	0.6667	0.2	0.3077
art-other	0.0	0.0	0.0
art-painting	0.0	0.0	0.0
art-writtenart	0.0	0.0	0.0
building-airport	0.7143	0.7692	0.7407
building-hospital	0.6667	0.7778	0.7179
building-hotel	0.7857	0.6875	0.7333
building-library	0.8182	0.75	0.7826
building-other	0.0	0.0	0.0
building-restaurant	0.8571	0.375	0.5217
building-sportsfacility	0.6667	0.5	0.5714
building-theater	0.9	0.5625	0.6923
datetime-absolute	0.3333	0.0769	0.125
datetime-authored	0.55	0.8462	0.6667
datetime-range	0.75	0.5	0.6
datetime-relative	0.0	0.0	0.0
event-attack/battle/war/militaryconflict	0.8	0.2857	0.4211
event-disaster	0.5385	0.5	0.5185
event-election	0.75	0.5	0.6
event-other	0.0	0.0	0.0
event-protest	0.5455	0.4615	0.5000
event-sportsevent	0.625	0.3846	0.4762
location-bodiesofwater	0.8333	0.3571	0.5
location-gpe	0.375	0.2143	0.2727
location-island	0.7143	0.3333	0.4545
location-mountain	0.5882	0.625	0.6061
location-other	0.0	0.0	0.0
location-park	0.6667	0.5	0.5714
location-road/railway/highway/transit	0.8	0.5333	0.64
organization-company	0.0	0.0	0.0
organization-education	0.3077	0.2857	0.2963
organization-government/governmentagency	0.25	0.0909	0.1333
organization-media/newspaper	0.5833	0.4667	0.5185
organization-other	1.0	0.0769	0.1429
organization-politicalparty	0.75	0.2727	0.4000
organization-religion	1.0	0.3077	0.4706
organization-showorganization	0.75	0.25	0.375
organization-sportsleague	0.8571	0.4286	0.5714
organization-sportsteam	0.4286	0.5	0.4615
other-astronomything	0.0	0.0	0.0
other-award	1.0	0.2143	0.3529
other-biologything	0.0	0.0	0.0
other-chemicalthing	0.4	0.3077	0.3478
other-currency	1.0	0.2143	0.3529
other-disease	0.5714	0.3077	0.4
other-educationaldegree	0.5833	0.5833	0.5833
other-god	0.8	0.2222	0.3478
other-language	0.8	0.2857	0.4211
other-law	0.6667	0.5	0.5714
other-livingthing	0.0	0.0	0.0
other-medical	0.0	0.0	0.0
person-actor	0.3448	0.5	0.4082
person-artist/author	0.6667	0.1429	0.2353
person-athlete	0.6667	0.2353	0.3478
person-director	0.2	0.0714	0.1053
person-other	0.0	0.0	0.0
person-politician	0.6667	0.0952	0.1667
person-scholar	0.4118	0.4667	0.4375
person-soldier	0.0	0.0	0.0
product-airplane	0.75	0.3333	0.4615
product-car	1.0	0.2143	0.3529
product-food	0.0	0.0	0.0
product-game	1.0	0.1333	0.2353
product-other	0.5	0.0909	0.1538
product-ship	0.75	0.3	0.4286
product-software	1.0	0.4167	0.5882
product-train	0.5556	0.3571	0.4348
product-weapon	0.3333	0.0625	0.1053

Uses

Direct Use for Inference

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("YurtsAI/named_entity_recognition_document_context")
# Run inference
entities = model.predict("We have Kanye West, Beyoncé, and Taylor Swift performing at the beachside park on the island of Maui.")

Downstream Use

You can finetune this model on your own dataset.

Click to expand

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("YurtsAI/named_entity_recognition_document_context")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("YurtsAI/named_entity_recognition_document_context-finetuned")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Sentence length	1	18.4126	309
Entities per sentence	0	0.9794	5

Training Hyperparameters

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3

Training Results

Epoch	Step	Validation Loss	Validation Precision	Validation Recall	Validation F1	Validation Accuracy
0.4322	500	0.0503	0.0	0.0	0.0	0.8898
0.8643	1000	0.0435	1.0	0.0010	0.0020	0.8900
1.2965	1500	0.0383	0.2841	0.0254	0.0466	0.8908
1.7286	2000	0.0326	0.5556	0.0710	0.1259	0.8951
2.1608	2500	0.0294	0.5806	0.1826	0.2778	0.9032
2.5929	3000	0.0278	0.6259	0.2698	0.3770	0.9109

Framework Versions

• Python: 3.12.2
• SpanMarker: 1.5.0
• Transformers: 4.41.2
• PyTorch: 2.3.1
• Datasets: 2.20.0
• Tokenizers: 0.19.1

Citation

BibTeX

@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}