Hugging Face's logo

lambdavi
/
span-marker-luke-legal

Token Classification
SpanMarker
Safetensors
ner
named-entity-recognition
generated_from_span_marker_trainer
legal
Eval Results
Model card Files Community
span-marker-luke-legal / README.md
lambdavi's picture
lambdavi
Update README.md
c856a7b verified
preview code
|
raw history blame contribute delete
No virus
9.83 kB
metadata
library_name: span-marker
tags:
  - span-marker
  - token-classification
  - ner
  - named-entity-recognition
  - generated_from_span_marker_trainer
  - legal
metrics:
  - precision
  - recall
  - f1
widget:
  - text: >-
      The seven-judge Constitution Bench of the Supreme Court in SBP and Co.
      (supra) while reversing earlier five-judge Constitution Bench judgment in
      Konkan Railway Corpn. Ltd. vs. Rani Construction (P) Ltd., (2002) 2 SCC
      388 held that the power exercised by the Chief Justice of the High Court
      or the Chief justice of India under Section 11(6) of the Arbitration Act
      is not an administrative power but is a judicial power.
  - text: >-
      In The High Court Of Judicature At Patna Criminal Writ Jurisdiction Case
      No.160 of 2021 Arising Out of Ps. Case No.-58 Year-2020 Thana- Bakhari
      District- Begusarai ======================================================
      Hanif Ur Rahman, son of Azhar Rahman, Resident of C-39, East Nizamuddin,
      New Delhi....... Petitioner Versus 1. The State of Bihar (through Chief
      Secretary, Govt. of Bihar) Main Secretariat, Patna - 800015. 2. Meena
      Khatoon, wife of Mastan @ Noor Mohammad, Resident of Village- Mansurpur
      Chaksikandar, P.S.- Bidupur, District- Vaishali (Bihar) 3. The Bihar
      Police, through Standing Counsel. 4. Child Welfare Committee, through
      Chairperson, Chanakyanagar, Mahmadpur, Begusarai. 5. The Superintendent,
      Alpawas Grih, Nirala Nagar, Behind G.D. College, Ratanpur,
      Begusarai....... Respondents
      ====================================================== Appearance:For the
      Petitioner:Ms. Kriti Awasthi, Advocate Mr. Sambhav Gupta, Advocate Mr.
      Navnit Kumar, Advocate Mr. Shyam Kumar, Advocate For the
      Respondents:Mr.Nadim Seraj, G.P.5 For the Resp. No. 2:Ms. Archana Sinha,
      Advocate For the Resp. No. 4:Mr. Prabhu Narain Sharma, Advocate
      ====================================================== Coram: Honourable
      Mr. Justice Rajeev Ranjan Prasad C.A.V. Judgment
  - text: >-
      1 R In The High Court Of Karnataka At Bengaluru Dated This The 19Th Day Of
      February, 2021 Before The Hon'Ble Mr. Justice H.P. Sandesh Criminal Appeal
      No.176/2011 Between: Sri G.L. Jagadish, S/O Sri G.N. Lingappa, Aged About
      52 Years, Residing At No.29, 3Rd Main, Basaveshwara Housing Society
      Layout, Vijayanagar, Near Bts Depot, Bengaluru-40....Appellant [By Sri H.
      Ramachandra, Advocate For Sri H.R. Anantha Krishna Murthy And Associates -
      (Through V.C.)] And: Smt. Vasantha Kokila, W/O Late N.R. Somashekhar, Aged
      About 58 Years, Residing At No.322, 8Th Main, 3Rd Stage, 4Th Block,
      Basaveshwaranagar, Bengaluru....Respondent [By Sri K.R. Lakshminarayana
      Rao, Advocate] This Criminal Appeal Is Filed Under Section 378(4) Of
      Cr.P.C. Praying To Set Aside The Order Dated 06.07.2010 Passed By The P.O.
      Ftc-Ii, Bengaluru In Crl.A. No.470/2009 And Confirming The Order Dated
      27.05.2009 Passed By The Xxii Acmm And Xxiv Ascj, Bengaluru In
      C.C.No.17229/2004 Convicting The Respondent/Accused For The Offence
      Punishable Under Section 138 Of Ni Act. 2 This Criminal Appeal Having Been
      Heard And Reserved For Orders On 06.02.2021 This Day, The Court Pronounced
      The Following: Judgment
  - text: >-
      The petition was filed through Sh. Vijay Pahwa, General Power of Attorney
      and it was asserted in the petition under Section 13-B of the Rent Act
      that 1 of 23 50% share of the demised premises had been purchased by the
      landlord from Sh. Vinod Malhotra vide sale deed No.4226 registered on
      20.12.2007 with Sub Registrar, Chandigarh.
  - text: >-
      Mr. Arun Bharadwaj, ld. CGSC, appearing for the Union of India, has
      Signature Not Verified Digitally Signed By:PRATHIBA M SINGH Signing
      Date:09.10.2020 16:15 Digitally Signed By:SINDHU KRISHNAKUMAR Signing
      Date:09.10.2020 16:50:02 reiterated the submissions made by Dr. Singhvi
      and has further submitted that this petition ought to be heard with the OA
      No. 291/138/2020 pending before the CAT.
pipeline_tag: token-classification
model-index:
  - name: SpanMarker
    results:
      - task:
          type: token-classification
          name: Named Entity Recognition
        dataset:
          name: legal_ner
          type: unknown
          split: eval
        metrics:
          - type: f1
            value: 0.9099756690997567
            name: F1
          - type: precision
            value: 0.9089703932832524
            name: Precision
          - type: recall
            value: 0.9109831709477414
            name: Recall

SpanMarker

This is a SpanMarker model that can be used for Named Entity Recognition. It was trained on the Legal NER Indian Justice dataset.

Model Details

Model Description

  • Model Type: SpanMarker
  • Maximum Sequence Length: 128 tokens
  • Maximum Entity Length: 6 words

Model Sources

Uses

Direct Use for Inference 

from span_marker import SpanMarkerModel
from span_marker.tokenizer import SpanMarkerTokenizer


# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("lambdavi/span-marker-luke-legal")
tokenizer = SpanMarkerTokenizer.from_pretrained("roberta-base", config=model.config)
model.set_tokenizer(tokenizer)

# Run inference
entities = model.predict("The petition was filed through Sh. Vijay Pahwa, General Power of Attorney and it was asserted in the petition under Section 13-B of the Rent Act that 1 of 23 50% share of the demised premises had been purchased by the landlord from Sh. Vinod Malhotra vide sale deed No.4226 registered on 20.12.2007 with Sub Registrar, Chandigarh.")

Downstream Use

You can finetune this model on your own dataset.

Click to expand 
from span_marker import SpanMarkerModel, Trainer
from span_marker.tokenizer import SpanMarkerTokenizer


# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("lambdavi/span-marker-luke-legal")
tokenizer = SpanMarkerTokenizer.from_pretrained("roberta-base", config=model.config)
model.set_tokenizer(tokenizer)

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("lambdavi/span-marker-luke-legal-finetuned")

Training Details

Training Set Metrics

Training set Min Median Max
Sentence length 3 44.5113 2795
Entities per sentence 0 2.7232 68

Training Hyperparameters

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.06
  • num_epochs: 5

Training Results

Epoch Step Validation Loss Validation Precision Validation Recall Validation F1 Validation Accuracy
0.9997 1837 0.0137 0.7773 0.7994 0.7882 0.9577
2.0 3675 0.0090 0.8751 0.8348 0.8545 0.9697
2.9997 5512 0.0077 0.8777 0.8959 0.8867 0.9770
4.0 7350 0.0061 0.8941 0.9083 0.9011 0.9811
4.9986 9185 0.0064 0.9090 0.9110 0.9100 0.9824
Metric Value
f1-exact 0.9237
f1-strict 0.9100
f1-partial 0.9365
f1-type-match 0.9277

Framework Versions

  • Python: 3.10.12
  • SpanMarker: 1.5.0
  • Transformers: 4.36.0
  • PyTorch: 2.0.0
  • Datasets: 2.17.1
  • Tokenizers: 0.15.0

Citation

BibTeX 

@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}