distilbert-base-uncased-finetuned-ner_finer_139

This model is a fine-tuned version of distilbert/distilbert-base-uncased on an nlpaueb/finer-139 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0019
• Precision: 0.8112
• Recall: 0.8934
• F1: 0.8503
• Accuracy: 0.9996

Model description

• Model Architecture: DistilBert (distilbert-base-uncased)
• Fine-tuning Details: The model was fine-tuned specifically for the NER task, adjusting the top layer to classify a set of entities defined in the FiNER-139 dataset.
• Parameters:
  • Top Labels Count: 4
  • Total Rows Processed: 100,000
  • Label All Tokens: True
  • Max Length: 200
  • Padding: Max Length
  • Truncation: True
  • Batch Size: 64

Dataset

• Source: nlpaueb/finer-139
• Size: A subset of 100,000 records was used from the dataset for training.
• Selected Labels (4 most frequent):
  • O
  • B-DebtInstrumentFaceAmount
  • I-DebtInstrumentFaceAmount
  • B-DebtInstrumentBasisSpreadOnVariableRate1
  • I-DebtInstrumentBasisSpreadOnVariableRate1
  • B-LesseeOperatingLeaseTermOfContract
  • I-LesseeOperatingLeaseTermOfContract
  • B-ContractWithCustomerLiability
  • I-ContractWithCustomerLiability

Training Process

The DistilBert model was trained using the specified subset of the FiNER-139 dataset. The training involved tokenizing the input text, aligning labels with tokens, and performing fine-tuning over 5 epochs. The learning rate was set to 2e-5 with a weight decay of 0.01 for regularization.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.0031	1.0	1094	0.0025	0.6633	0.8234	0.7347	0.9992
0.002	2.0	2188	0.0018	0.8088	0.8224	0.8155	0.9995
0.0012	3.0	3282	0.0018	0.7680	0.8924	0.8255	0.9995
0.0008	4.0	4376	0.0017	0.8479	0.8736	0.8605	0.9996
0.0005	5.0	5470	0.0019	0.8112	0.8934	0.8503	0.9996

Evaluation Results per Class

In addition to overall metrics, the model's performance was assessed for each entity type. Below are the evaluation results per class, providing insight into the model's ability to accurately identify and classify various named entities:

Entity Type	Precision	Recall	F1-Score	Support
ContractWithCustomerLiability	0.895	0.607	0.723	28
DebtInstrumentBasisSpreadOnVariableRate1	0.866	0.969	0.914	452
DebtInstrumentFaceAmount	0.754	0.844	0.797	469
LesseeOperatingLeaseTermOfContract	1.000	0.500	0.667	8

Example of usage

Example of usage can be found here

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2