metadata

library_name: transformers
license: apache-2.0
base_model: bert-base-multilingual-cased
tags:
  - generated_from_trainer
model-index:
  - name: fine_tune_bert_output
    results: []

fine_tune_bert_output

This model is a fine-tuned version of bert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.3320
Overall Precision: 0.9051
Overall Recall: 0.9121
Overall F1: 0.9086
Overall Accuracy: 0.9577
Loc F1: 0.9190
Org F1: 0.8663
Per F1: 0.9367

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Overall Precision	Overall Recall	Overall F1	Overall Accuracy	Loc F1	Org F1	Per F1
0.2713	0.8	1000	0.2236	0.8498	0.8672	0.8584	0.9401	0.8834	0.8019	0.8790
0.1537	1.6	2000	0.1909	0.8772	0.8943	0.8857	0.9495	0.9002	0.8369	0.9164
0.1152	2.4	3000	0.2095	0.8848	0.8981	0.8914	0.9523	0.9039	0.8432	0.9220
0.0889	3.2	4000	0.2223	0.8978	0.8998	0.8988	0.9546	0.9080	0.8569	0.9290
0.0701	4.0	5000	0.2152	0.8937	0.9042	0.8989	0.9544	0.9113	0.8565	0.9246
0.0457	4.8	6000	0.2365	0.9017	0.9069	0.9043	0.9563	0.9164	0.8616	0.9310
0.0364	5.6	7000	0.2622	0.9037	0.9086	0.9061	0.9578	0.9148	0.8639	0.9365
0.026	6.4	8000	0.2916	0.9037	0.9159	0.9097	0.9585	0.9183	0.8712	0.9366
0.0215	7.2	9000	0.2985	0.9022	0.9128	0.9074	0.9565	0.9178	0.8676	0.9323
0.0134	8.0	10000	0.3071	0.904	0.9131	0.9085	0.9574	0.9198	0.8671	0.9344
0.0091	8.8	11000	0.3335	0.9056	0.9115	0.9085	0.9573	0.9175	0.8670	0.9373
0.0074	9.6	12000	0.3320	0.9051	0.9121	0.9086	0.9577	0.9190	0.8663	0.9367

Framework versions

Transformers 4.44.2
Pytorch 2.4.1+cu121
Datasets 3.2.0
Tokenizers 0.19.1