Rishabh157/spanmarker-multiconer-mdeberta

Multilingual SpanMarker NER: mDeBERTa-v3 on MultiCoNER 2023 (33 Fine-Grained Classes, 10 Languages)

Developed by: Rishabh Kumar
Model Name: Rishabh157/spanmarker-multiconer-mdeberta
Base Architecture: microsoft/mdeberta-v3-base

This repository provides an enterprise-grade, state-of-the-art multilingual Named Entity Recognition (NER) model developed by Rishabh Kumar, combining microsoft/mdeberta-v3-base with the SpanMarker candidate span-classification framework, fine-tuned on MultiCoNER 2023 (SemEval-2023 Task 2) across 10 languages for 6 full epochs (117,756 steps).

---

Model Overview

• Developer / Creator: Rishabh Kumar (@Rishabh157)
• Base Backbone: microsoft/mdeberta-v3-base (12 layers, 768 hidden dimension, disentangled relative positional attention).
• Task Formulation: SpanMarker (Candidate phrase classification with boundary marker tokens [S_i, E_j] and Greedy Non-Maximum Suppression).
• Tuning Strategy: Full-Parameter Fine-Tuning (all 278.8M parameters active).
• Sequence Parameters:
  • model_max_length: 384 tokens (100% empirical coverage across multi-sentence titles).
  • entity_max_length: 16 words (captures 99.99% of complex, multi-word entities).
  • marker_max_length: 128 candidate tokens.
• Supported Languages (10 Languages):
  • Bangla (bn), German (de), English (en), Spanish (es), French (fr), Hindi (hi), Italian (it), Multilingual (multi), Portuguese (pt), Swedish (sv).
• Entity Taxonomy: 33 Fine-Grained Classes (67 BIO tags):
  • Person Subtypes: Artist, Athlete, Cleric, Politician, Scientist, SportsManager, OtherPER
  • Location Subtypes: Facility, HumanSettlement, Station, OtherLOC
  • Organization Subtypes: AerospaceManufacturer, CarManufacturer, MusicalGRP, PublicCorp, PrivateCorp, SportsGRP, ORG
  • Creative & Visual Works: ArtWork, MusicalWork, VisualWork, WrittenWork
  • Products & Technology: Clothing, Drink, Food, Software, Vehicle, OtherPROD
  • Biomedical & Healthcare: AnatomicalStructure, Disease, MedicalProcedure, Medication/Vaccine, Symptom

---

Empirical Benchmark Performance

Evaluated on the full official multilingual MultiCoNER 2023 validation set (15,578 sentences across all 10 language splits) using strict exact-span seqeval matching:

Overall 6-Epoch Validation Results

Metric	Score	Notes
Overall F1 Score	66.97%	+13.69% higher than previous standalone Token-LoRA (53.28%)
Extraction Precision	73.87%	+20.77% higher than previous standalone Token-LoRA (53.10%)
Extraction Recall	61.25%	Balanced boundary discovery across complex, low-context spans
Sequence Accuracy	90.39%	Exact token and span boundary classification
Validation Loss	0.0234	Consistent, monotonic drop across all 6 epochs

---

Comparison with Prior Benchmarks

Model	Architecture	Taxonomy Scope	Precision	Recall	F1 Score
Token-LoRA v1	r=16, Attn-only	MultiCoNER 33 classes	46.26%	51.79%	48.87%
Token-LoRA v2	r=32, Attn+FFN, naive weights	MultiCoNER 33 classes	17.74%	48.41%	25.97%
Token-LoRA v3 (Prev. Best)	r=32, Attn+FFN, unweighted	MultiCoNER 33 classes	53.10%	53.47%	53.28%
Master Unified Token-LoRA	r=64, 1.53M joint dataset	95-tag joint taxonomy	59.57%	59.23%	59.37%
SpanMarker (Epoch 6.0) 🏆	Full FT, SpanMarker	MultiCoNER 33 classes	73.87%	61.25%	66.97%

---

Per-Class Detailed Performance Breakdown (33 Categories)

Entity Class	Precision	Recall	F1 Score	Support Count
HumanSettlement	87.00%	76.99%	81.69%	2,903
AerospaceManufacturer	82.14%	75.41%	78.63%	244
Drink	77.97%	73.90%	75.88%	249
Symptom	80.93%	69.78%	74.94%	225
PrivateCorp	84.57%	66.07%	74.19%	224
Artist	77.37%	69.87%	73.43%	3,077
SportsGRP	77.99%	67.01%	72.08%	682
PublicCorp	76.60%	67.29%	71.64%	428
CarManufacturer	72.93%	69.04%	70.93%	281
Athlete	74.76%	65.80%	69.99%	1,301
Software	79.11%	62.52%	69.84%	539
AnatomicalStructure	71.93%	67.18%	69.47%	393
Disease	72.20%	66.67%	69.33%	339
MusicalGRP	79.29%	60.09%	68.37%	669
Medication/Vaccine	69.75%	66.47%	68.07%	340
OtherLOC	80.07%	58.23%	67.43%	407
VisualWork	75.82%	58.22%	65.87%	1,228
Station	72.30%	60.36%	65.79%	333
SportsManager	68.73%	62.61%	65.53%	337
WrittenWork	73.91%	57.00%	64.37%	1,014
MedicalProcedure	71.22%	58.63%	64.32%	249
MusicalWork	75.51%	54.56%	63.35%	746
Facility	72.18%	54.98%	62.42%	873
Food	63.44%	56.91%	60.00%	369
ORG	71.13%	51.27%	59.59%	1,182
Cleric	64.41%	53.52%	58.46%	355
Politician	64.03%	52.84%	57.90%	1,058
ArtWork	77.94%	45.69%	57.61%	232
OtherPROD	64.65%	48.15%	55.19%	756
Vehicle	57.74%	50.56%	53.92%	354
Clothing	61.99%	47.53%	53.81%	223
OtherPER	55.56%	41.01%	47.19%	1,524
Scientist	47.80%	43.12%	45.34%	327

Note on "Support Count": In Named Entity Recognition benchmarks, Support indicates the exact number of ground-truth entity spans belonging to that category in the validation dataset (15,578 sentences). It represents the sample size over which Recall and F1 are computed, providing transparency into entity frequency across the corpus.

---

Epoch-by-Epoch Training Trajectory

Epoch	Step	Eval Loss	Precision	Recall	F1 Score	Accuracy
1.0	19,626	0.0482	54.78%	7.28%	12.85%	81.31%
2.0	39,252	0.0357	66.27%	25.28%	36.60%	83.66%
3.0	58,878	0.0295	71.30%	38.80%	50.26%	86.29%
4.0	78,504	0.0273	71.16%	47.10%	56.68%	87.73%
6.0	117,756	0.0234	73.87%	61.25%	66.97% 🏆	90.39%

---

Usage Example

from span_marker import SpanMarkerModel

# Load the model directly from Hugging Face Hub
model = SpanMarkerModel.from_pretrained("Rishabh157/spanmarker-multiconer-mdeberta")

# Multilingual test sentences
sentences = [
    "Boeing and Airbus compete directly in the commercial aircraft market.",
    "Der FC Bayern München gewann das Finale der UEFA Champions League.",
    "El presidente de Francia anunció nuevas medidas en París.",
    "डॉ. एपीजे अब्दुल कलाम भारत के राष्ट्रपति और एक प्रसिद्ध वैज्ञानिक थे।"
]

# Run span-level entity inference
entities = model.predict(sentences)

for sent, ent_list in zip(sentences, entities):
    print(f"\nSentence: {sent}")
    for ent in ent_list:
        print(f"  [{ent['label']}] -> '{ent['char_span']}' (span: {ent['word']})")

---

Training Configuration & Reproducibility

• Optimizer: AdamW (β₁=0.9, β₂=0.999, ε=1e-8)
• Learning Rate: 3.0e-5 with Cosine Annealing Schedule
• Warmup Ratio: 10% (first 11,775 steps)
• Batch Size: 16 per device (Effective batch size = 16)
• Weight Decay: 0.01
• Gradient Clipping: Max norm 1.0
• Frameworks: transformers 4.41+, span-marker 1.5.0, PyTorch 2.3+

---

Author & Citation

This model was trained, evaluated, and published by Rishabh Kumar (@Rishabh157).

If you use this model or refer to this work, please cite:

@misc{rishabhkumar2026_spanmarker_multiconer,
  author = {Rishabh Kumar},
  title = {SpanMarker Fine-Grained Multilingual NER with mDeBERTa-v3 on MultiCoNER 2023},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Rishabh157/spanmarker-multiconer-mdeberta}}
}