metadata

library_name: transformers
license: apache-2.0
base_model: answerdotai/ModernBERT-base
tags:
  - generated_from_trainer
model-index:
  - name: ModernBERT-base-iob2-20241223160124
    results: []

ModernBERT-base-iob2-20241223160124

This model is a fine-tuned version of answerdotai/ModernBERT-base on an unknown dataset. It achieves the following results on the evaluation set:

eval_loss: 4.5330
eval_model_preparation_time: 0.0027
eval_overall_strict_precision: 0.0014
eval_overall_strict_recall: 0.0001
eval_overall_strict_f1: 0.0001
eval_overall_ent_type_precision: 0.0057
eval_overall_ent_type_recall: 0.0003
eval_overall_ent_type_f1: 0.0005
eval_overall_partial_precision: 0.2577
eval_overall_partial_recall: 0.0130
eval_overall_partial_f1: 0.0247
eval_overall_exact_precision: 0.1638
eval_overall_exact_recall: 0.0082
eval_overall_exact_f1: 0.0157
eval_checkOut_strict_precision: 0.0
eval_checkOut_strict_recall: 0.0
eval_checkOut_strict_f1: 0
eval_checkOut_ent_type_precision: 0.0
eval_checkOut_ent_type_recall: 0.0
eval_checkOut_ent_type_f1: 0
eval_checkOut_partial_precision: 0.0030
eval_checkOut_partial_recall: 0.0031
eval_checkOut_partial_f1: 0.0030
eval_checkOut_exact_precision: 0.0022
eval_checkOut_exact_recall: 0.0023
eval_checkOut_exact_f1: 0.0023
eval_bookingNumber_strict_precision: 0.0
eval_bookingNumber_strict_recall: 0.0
eval_bookingNumber_strict_f1: 0
eval_bookingNumber_ent_type_precision: 0.0
eval_bookingNumber_ent_type_recall: 0.0
eval_bookingNumber_ent_type_f1: 0
eval_bookingNumber_partial_precision: 0.0015
eval_bookingNumber_partial_recall: 0.0067
eval_bookingNumber_partial_f1: 0.0024
eval_bookingNumber_exact_precision: 0.0014
eval_bookingNumber_exact_recall: 0.0063
eval_bookingNumber_exact_f1: 0.0023
eval_documentType_strict_precision: 0.0
eval_documentType_strict_recall: 0.0
eval_documentType_strict_f1: 0
eval_documentType_ent_type_precision: 0.0006
eval_documentType_ent_type_recall: 0.0001
eval_documentType_ent_type_f1: 0.0001
eval_documentType_partial_precision: 0.1126
eval_documentType_partial_recall: 0.0171
eval_documentType_partial_f1: 0.0297
eval_documentType_exact_precision: 0.0816
eval_documentType_exact_recall: 0.0124
eval_documentType_exact_f1: 0.0215
eval_companyCountry_strict_precision: 0.0010
eval_companyCountry_strict_recall: 0.0003
eval_companyCountry_strict_f1: 0.0005
eval_companyCountry_ent_type_precision: 0.0014
eval_companyCountry_ent_type_recall: 0.0005
eval_companyCountry_ent_type_f1: 0.0008
eval_companyCountry_partial_precision: 0.0455
eval_companyCountry_partial_recall: 0.0167
eval_companyCountry_partial_f1: 0.0245
eval_companyCountry_exact_precision: 0.0204
eval_companyCountry_exact_recall: 0.0075
eval_companyCountry_exact_f1: 0.0110
eval_hotelName_strict_precision: 0.0
eval_hotelName_strict_recall: 0.0
eval_hotelName_strict_f1: 0
eval_hotelName_ent_type_precision: 0.0
eval_hotelName_ent_type_recall: 0.0
eval_hotelName_ent_type_f1: 0
eval_hotelName_partial_precision: 0.0003
eval_hotelName_partial_recall: 0.0016
eval_hotelName_partial_f1: 0.0005
eval_hotelName_exact_precision: 0.0003
eval_hotelName_exact_recall: 0.0014
eval_hotelName_exact_f1: 0.0005
eval_hotelBankAccount_strict_precision: 0.0
eval_hotelBankAccount_strict_recall: 0.0
eval_hotelBankAccount_strict_f1: 0
eval_hotelBankAccount_ent_type_precision: 0.0
eval_hotelBankAccount_ent_type_recall: 0.0
eval_hotelBankAccount_ent_type_f1: 0
eval_hotelBankAccount_partial_precision: 0.0
eval_hotelBankAccount_partial_recall: 0.0
eval_hotelBankAccount_partial_f1: 0
eval_hotelBankAccount_exact_precision: 0.0
eval_hotelBankAccount_exact_recall: 0.0
eval_hotelBankAccount_exact_f1: 0
eval_hotelAddress_strict_precision: 0.0
eval_hotelAddress_strict_recall: 0.0
eval_hotelAddress_strict_f1: 0
eval_hotelAddress_ent_type_precision: 0.0
eval_hotelAddress_ent_type_recall: 0.0
eval_hotelAddress_ent_type_f1: 0
eval_hotelAddress_partial_precision: 0.0
eval_hotelAddress_partial_recall: 0.0
eval_hotelAddress_partial_f1: 0
eval_hotelAddress_exact_precision: 0.0
eval_hotelAddress_exact_recall: 0.0
eval_hotelAddress_exact_f1: 0
eval_companyZipcode_strict_precision: 0.0
eval_companyZipcode_strict_recall: 0.0
eval_companyZipcode_strict_f1: 0
eval_companyZipcode_ent_type_precision: 0.0
eval_companyZipcode_ent_type_recall: 0.0
eval_companyZipcode_ent_type_f1: 0
eval_companyZipcode_partial_precision: 0.0005
eval_companyZipcode_partial_recall: 0.0028
eval_companyZipcode_partial_f1: 0.0008
eval_companyZipcode_exact_precision: 0.0004
eval_companyZipcode_exact_recall: 0.0026
eval_companyZipcode_exact_f1: 0.0007
eval_companyAddress_strict_precision: 0.0
eval_companyAddress_strict_recall: 0.0
eval_companyAddress_strict_f1: 0
eval_companyAddress_ent_type_precision: 0.0
eval_companyAddress_ent_type_recall: 0.0
eval_companyAddress_ent_type_f1: 0
eval_companyAddress_partial_precision: 0.0001
eval_companyAddress_partial_recall: 0.0038
eval_companyAddress_partial_f1: 0.0001
eval_companyAddress_exact_precision: 0.0001
eval_companyAddress_exact_recall: 0.0038
eval_companyAddress_exact_f1: 0.0001
eval_netAmount_strict_precision: 0.0
eval_netAmount_strict_recall: 0.0
eval_netAmount_strict_f1: 0
eval_netAmount_ent_type_precision: 0.0
eval_netAmount_ent_type_recall: 0.0
eval_netAmount_ent_type_f1: 0
eval_netAmount_partial_precision: 0.0117
eval_netAmount_partial_recall: 0.0036
eval_netAmount_partial_f1: 0.0055
eval_netAmount_exact_precision: 0.0048
eval_netAmount_exact_recall: 0.0015
eval_netAmount_exact_f1: 0.0023
eval_hotelCountry_strict_precision: 0.0
eval_hotelCountry_strict_recall: 0.0
eval_hotelCountry_strict_f1: 0
eval_hotelCountry_ent_type_precision: 0.0
eval_hotelCountry_ent_type_recall: 0.0
eval_hotelCountry_ent_type_f1: 0
eval_hotelCountry_partial_precision: 0.0015
eval_hotelCountry_partial_recall: 0.0017
eval_hotelCountry_partial_f1: 0.0016
eval_hotelCountry_exact_precision: 0.0015
eval_hotelCountry_exact_recall: 0.0017
eval_hotelCountry_exact_f1: 0.0016
eval_cardNumber_strict_precision: 0.0
eval_cardNumber_strict_recall: 0.0
eval_cardNumber_strict_f1: 0
eval_cardNumber_ent_type_precision: 0.0
eval_cardNumber_ent_type_recall: 0.0
eval_cardNumber_ent_type_f1: 0
eval_cardNumber_partial_precision: 0.0020
eval_cardNumber_partial_recall: 0.0010
eval_cardNumber_partial_f1: 0.0013
eval_cardNumber_exact_precision: 0.0020
eval_cardNumber_exact_recall: 0.0010
eval_cardNumber_exact_f1: 0.0013
eval_cardType_strict_precision: 0.0
eval_cardType_strict_recall: 0.0
eval_cardType_strict_f1: 0
eval_cardType_ent_type_precision: 0.0001
eval_cardType_ent_type_recall: 0.0007
eval_cardType_ent_type_f1: 0.0002
eval_cardType_partial_precision: 0.0008
eval_cardType_partial_recall: 0.0050
eval_cardType_partial_f1: 0.0014
eval_cardType_exact_precision: 0.0005
eval_cardType_exact_recall: 0.0032
eval_cardType_exact_f1: 0.0009
eval_grossAmount_strict_precision: 0.0
eval_grossAmount_strict_recall: 0.0
eval_grossAmount_strict_f1: 0
eval_grossAmount_ent_type_precision: 0.0
eval_grossAmount_ent_type_recall: 0.0
eval_grossAmount_ent_type_f1: 0
eval_grossAmount_partial_precision: 0.0001
eval_grossAmount_partial_recall: 0.0014
eval_grossAmount_partial_f1: 0.0001
eval_grossAmount_exact_precision: 0.0001
eval_grossAmount_exact_recall: 0.0014
eval_grossAmount_exact_f1: 0.0001
eval_reservationNumber_strict_precision: 0.0
eval_reservationNumber_strict_recall: 0.0
eval_reservationNumber_strict_f1: 0
eval_reservationNumber_ent_type_precision: 0.0
eval_reservationNumber_ent_type_recall: 0.0
eval_reservationNumber_ent_type_f1: 0
eval_reservationNumber_partial_precision: 0.0008
eval_reservationNumber_partial_recall: 0.0045
eval_reservationNumber_partial_f1: 0.0014
eval_reservationNumber_exact_precision: 0.0008
eval_reservationNumber_exact_recall: 0.0045
eval_reservationNumber_exact_f1: 0.0014
eval_invoiceNumber_strict_precision: 0.0007
eval_invoiceNumber_strict_recall: 0.0002
eval_invoiceNumber_strict_f1: 0.0003
eval_invoiceNumber_ent_type_precision: 0.0011
eval_invoiceNumber_ent_type_recall: 0.0003
eval_invoiceNumber_ent_type_f1: 0.0005
eval_invoiceNumber_partial_precision: 0.0461
eval_invoiceNumber_partial_recall: 0.0135
eval_invoiceNumber_partial_f1: 0.0209
eval_invoiceNumber_exact_precision: 0.0118
eval_invoiceNumber_exact_recall: 0.0035
eval_invoiceNumber_exact_f1: 0.0054
eval_hotelVATNumber_strict_precision: 0.0001
eval_hotelVATNumber_strict_recall: 0.0005
eval_hotelVATNumber_strict_f1: 0.0001
eval_hotelVATNumber_ent_type_precision: 0.0001
eval_hotelVATNumber_ent_type_recall: 0.0005
eval_hotelVATNumber_ent_type_f1: 0.0001
eval_hotelVATNumber_partial_precision: 0.0004
eval_hotelVATNumber_partial_recall: 0.0035
eval_hotelVATNumber_partial_f1: 0.0008
eval_hotelVATNumber_exact_precision: 0.0004
eval_hotelVATNumber_exact_recall: 0.0035
eval_hotelVATNumber_exact_f1: 0.0008
eval_externalReservationNumber_strict_precision: 0.0
eval_externalReservationNumber_strict_recall: 0.0
eval_externalReservationNumber_strict_f1: 0
eval_externalReservationNumber_ent_type_precision: 0.0
eval_externalReservationNumber_ent_type_recall: 0.0
eval_externalReservationNumber_ent_type_f1: 0
eval_externalReservationNumber_partial_precision: 0.0000
eval_externalReservationNumber_partial_recall: 0.0012
eval_externalReservationNumber_partial_f1: 0.0001
eval_externalReservationNumber_exact_precision: 0.0
eval_externalReservationNumber_exact_recall: 0.0
eval_externalReservationNumber_exact_f1: 0
eval_hotelFaxNumber_strict_precision: 0.0
eval_hotelFaxNumber_strict_recall: 0.0
eval_hotelFaxNumber_strict_f1: 0
eval_hotelFaxNumber_ent_type_precision: 0.0
eval_hotelFaxNumber_ent_type_recall: 0.0
eval_hotelFaxNumber_ent_type_f1: 0
eval_hotelFaxNumber_partial_precision: 0.0001
eval_hotelFaxNumber_partial_recall: 0.0031
eval_hotelFaxNumber_partial_f1: 0.0001
eval_hotelFaxNumber_exact_precision: 0.0
eval_hotelFaxNumber_exact_recall: 0.0
eval_hotelFaxNumber_exact_f1: 0
eval_roomNo_strict_precision: 0.0001
eval_roomNo_strict_recall: 0.0017
eval_roomNo_strict_f1: 0.0001
eval_roomNo_ent_type_precision: 0.0001
eval_roomNo_ent_type_recall: 0.0017
eval_roomNo_ent_type_f1: 0.0001
eval_roomNo_partial_precision: 0.0001
eval_roomNo_partial_recall: 0.0025
eval_roomNo_partial_f1: 0.0002
eval_roomNo_exact_precision: 0.0001
eval_roomNo_exact_recall: 0.0017
eval_roomNo_exact_f1: 0.0001
eval_companyName_strict_precision: 0.0002
eval_companyName_strict_recall: 0.0001
eval_companyName_strict_f1: 0.0001
eval_companyName_ent_type_precision: 0.0040
eval_companyName_ent_type_recall: 0.0017
eval_companyName_ent_type_f1: 0.0024
eval_companyName_partial_precision: 0.0422
eval_companyName_partial_recall: 0.0179
eval_companyName_partial_f1: 0.0252
eval_companyName_exact_precision: 0.0244
eval_companyName_exact_recall: 0.0104
eval_companyName_exact_f1: 0.0145
eval_hotelEmail_strict_precision: 0.0
eval_hotelEmail_strict_recall: 0.0
eval_hotelEmail_strict_f1: 0
eval_hotelEmail_ent_type_precision: 0.0
eval_hotelEmail_ent_type_recall: 0.0
eval_hotelEmail_ent_type_f1: 0
eval_hotelEmail_partial_precision: 0.0024
eval_hotelEmail_partial_recall: 0.0199
eval_hotelEmail_partial_f1: 0.0043
eval_hotelEmail_exact_precision: 0.0024
eval_hotelEmail_exact_recall: 0.0199
eval_hotelEmail_exact_f1: 0.0043
eval_companyVATNumber_strict_precision: 0.0
eval_companyVATNumber_strict_recall: 0.0
eval_companyVATNumber_strict_f1: 0
eval_companyVATNumber_ent_type_precision: 0.0
eval_companyVATNumber_ent_type_recall: 0.0
eval_companyVATNumber_ent_type_f1: 0
eval_companyVATNumber_partial_precision: 0.0020
eval_companyVATNumber_partial_recall: 0.0020
eval_companyVATNumber_partial_f1: 0.0020
eval_companyVATNumber_exact_precision: 0.0018
eval_companyVATNumber_exact_recall: 0.0017
eval_companyVATNumber_exact_f1: 0.0018
eval_invoiceDate_strict_precision: 0.0
eval_invoiceDate_strict_recall: 0.0
eval_invoiceDate_strict_f1: 0
eval_invoiceDate_ent_type_precision: 0.0
eval_invoiceDate_ent_type_recall: 0.0
eval_invoiceDate_ent_type_f1: 0
eval_invoiceDate_partial_precision: 0.0058
eval_invoiceDate_partial_recall: 0.0127
eval_invoiceDate_partial_f1: 0.0080
eval_invoiceDate_exact_precision: 0.0035
eval_invoiceDate_exact_recall: 0.0077
eval_invoiceDate_exact_f1: 0.0048
eval_companyCity_strict_precision: 0.0
eval_companyCity_strict_recall: 0.0
eval_companyCity_strict_f1: 0
eval_companyCity_ent_type_precision: 0.0
eval_companyCity_ent_type_recall: 0.0
eval_companyCity_ent_type_f1: 0
eval_companyCity_partial_precision: 0.0012
eval_companyCity_partial_recall: 0.0053
eval_companyCity_partial_f1: 0.0020
eval_companyCity_exact_precision: 0.0010
eval_companyCity_exact_recall: 0.0044
eval_companyCity_exact_f1: 0.0017
eval_hotelPhoneNumber_strict_precision: 0.0
eval_hotelPhoneNumber_strict_recall: 0.0
eval_hotelPhoneNumber_strict_f1: 0
eval_hotelPhoneNumber_ent_type_precision: 0.0
eval_hotelPhoneNumber_ent_type_recall: 0.0
eval_hotelPhoneNumber_ent_type_f1: 0
eval_hotelPhoneNumber_partial_precision: 0.0006
eval_hotelPhoneNumber_partial_recall: 0.0047
eval_hotelPhoneNumber_partial_f1: 0.0011
eval_hotelPhoneNumber_exact_precision: 0.0004
eval_hotelPhoneNumber_exact_recall: 0.0031
eval_hotelPhoneNumber_exact_f1: 0.0007
eval_hotelTaxCode_strict_precision: 0.0
eval_hotelTaxCode_strict_recall: 0.0
eval_hotelTaxCode_strict_f1: 0
eval_hotelTaxCode_ent_type_precision: 0.0
eval_hotelTaxCode_ent_type_recall: 0.0
eval_hotelTaxCode_ent_type_f1: 0
eval_hotelTaxCode_partial_precision: 0.0007
eval_hotelTaxCode_partial_recall: 0.0019
eval_hotelTaxCode_partial_f1: 0.0010
eval_hotelTaxCode_exact_precision: 0.0007
eval_hotelTaxCode_exact_recall: 0.0019
eval_hotelTaxCode_exact_f1: 0.0010
eval_travellerName_strict_precision: 0.0
eval_travellerName_strict_recall: 0.0
eval_travellerName_strict_f1: 0
eval_travellerName_ent_type_precision: 0.0
eval_travellerName_ent_type_recall: 0.0
eval_travellerName_ent_type_f1: 0
eval_travellerName_partial_precision: 0.0010
eval_travellerName_partial_recall: 0.0157
eval_travellerName_partial_f1: 0.0019
eval_travellerName_exact_precision: 0.0010
eval_travellerName_exact_recall: 0.0157
eval_travellerName_exact_f1: 0.0019
eval_hotelCity_strict_precision: 0.0
eval_hotelCity_strict_recall: 0.0
eval_hotelCity_strict_f1: 0
eval_hotelCity_ent_type_precision: 0.0001
eval_hotelCity_ent_type_recall: 0.0002
eval_hotelCity_ent_type_f1: 0.0001
eval_hotelCity_partial_precision: 0.0044
eval_hotelCity_partial_recall: 0.0140
eval_hotelCity_partial_f1: 0.0067
eval_hotelCity_exact_precision: 0.0035
eval_hotelCity_exact_recall: 0.0111
eval_hotelCity_exact_f1: 0.0053
eval_checkIn_strict_precision: 0.0
eval_checkIn_strict_recall: 0.0
eval_checkIn_strict_f1: 0
eval_checkIn_ent_type_precision: 0.0
eval_checkIn_ent_type_recall: 0.0
eval_checkIn_ent_type_f1: 0
eval_checkIn_partial_precision: 0.0002
eval_checkIn_partial_recall: 0.0009
eval_checkIn_partial_f1: 0.0003
eval_checkIn_exact_precision: 0.0001
eval_checkIn_exact_recall: 0.0004
eval_checkIn_exact_f1: 0.0001
eval_currencyCode_strict_precision: 0.0
eval_currencyCode_strict_recall: 0.0
eval_currencyCode_strict_f1: 0
eval_currencyCode_ent_type_precision: 0.0
eval_currencyCode_ent_type_recall: 0.0
eval_currencyCode_ent_type_f1: 0
eval_currencyCode_partial_precision: 0.0001
eval_currencyCode_partial_recall: 0.0018
eval_currencyCode_partial_f1: 0.0002
eval_currencyCode_exact_precision: 0.0
eval_currencyCode_exact_recall: 0.0
eval_currencyCode_exact_f1: 0
eval_pageNumber_strict_precision: 0.0
eval_pageNumber_strict_recall: 0.0
eval_pageNumber_strict_f1: 0
eval_pageNumber_ent_type_precision: 0.0
eval_pageNumber_ent_type_recall: 0.0
eval_pageNumber_ent_type_f1: 0
eval_pageNumber_partial_precision: 0.0005
eval_pageNumber_partial_recall: 0.0070
eval_pageNumber_partial_f1: 0.0009
eval_pageNumber_exact_precision: 0.0005
eval_pageNumber_exact_recall: 0.0070
eval_pageNumber_exact_f1: 0.0009
eval_hotelZipCode_strict_precision: 0.0
eval_hotelZipCode_strict_recall: 0.0
eval_hotelZipCode_strict_f1: 0
eval_hotelZipCode_ent_type_precision: 0.0
eval_hotelZipCode_ent_type_recall: 0.0
eval_hotelZipCode_ent_type_f1: 0
eval_hotelZipCode_partial_precision: 0.0008
eval_hotelZipCode_partial_recall: 0.0039
eval_hotelZipCode_partial_f1: 0.0013
eval_hotelZipCode_exact_precision: 0.0006
eval_hotelZipCode_exact_recall: 0.0031
eval_hotelZipCode_exact_f1: 0.0010
eval_taxAmount_strict_precision: 0.0
eval_taxAmount_strict_recall: 0.0
eval_taxAmount_strict_f1: 0
eval_taxAmount_ent_type_precision: 0.0008
eval_taxAmount_ent_type_recall: 0.0004
eval_taxAmount_ent_type_f1: 0.0005
eval_taxAmount_partial_precision: 0.0723
eval_taxAmount_partial_recall: 0.0353
eval_taxAmount_partial_f1: 0.0474
eval_taxAmount_exact_precision: 0.0608
eval_taxAmount_exact_recall: 0.0296
eval_taxAmount_exact_f1: 0.0399
eval_runtime: 24.6705
eval_samples_per_second: 40.413
eval_steps_per_second: 1.297
step: 0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 16
total_train_batch_size: 512
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.5
num_epochs: 8

Framework versions

Transformers 4.48.0.dev0
Pytorch 2.3.1
Datasets 3.2.0
Tokenizers 0.21.0