This is a model card copied from original Tensorflow model version: https://huggingface.co/fimu-docproc-research/mbert-finetuned

xmelus/mbert

This model is a fine-tuned version of bert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 1.5424
  • Train Accuracy: 0.1446
  • Validation Loss: 1.5269
  • Validation Accuracy: 0.1461
  • Finished epochs: 24

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-05, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': -596, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'passive_serialization': True}, 'warmup_steps': 1000, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
  • training_precision: mixed_float16

Training results

Epoch 1/50

loss: 2.9925 - accuracy: 0.1059 - val_loss: 1.9812 - val_accuracy: 0.1331

Epoch 2/50

loss: 1.9979 - accuracy: 0.1307 - val_loss: 1.6063 - val_accuracy: 0.1429

Epoch 3/50

loss: 1.5798 - accuracy: 0.1434 - val_loss: 1.5332 - val_accuracy: 0.1461

Epoch 4/50

loss: 1.5325 - accuracy: 0.1451 - val_loss: 1.5285 - val_accuracy: 0.1458

Epoch 5/50

loss: 1.5415 - accuracy: 0.1448 - val_loss: 1.5449 - val_accuracy: 0.1457

Epoch 6/50

loss: 1.5395 - accuracy: 0.1448 - val_loss: 1.5448 - val_accuracy: 0.1456

Epoch 7/50

loss: 1.5463 - accuracy: 0.1446 - val_loss: 1.5421 - val_accuracy: 0.1454

Epoch 8/50

loss: 1.5352 - accuracy: 0.1451 - val_loss: 1.5536 - val_accuracy: 0.1453

Epoch 9/50

oss: 1.5230 - accuracy: 0.1451 - val_loss: 1.5097 - val_accuracy: 0.1466

Epoch 10/50

loss: 1.5318 - accuracy: 0.1449 - val_loss: 1.5303 - val_accuracy: 0.1460

Epoch 11/50

loss: 1.5364 - accuracy: 0.1448 - val_loss: 1.5280 - val_accuracy: 0.1462

Epoch 12/50

loss: 1.5411 - accuracy: 0.1444 - val_loss: 1.5493 - val_accuracy: 0.1455

Epoch 13/50

loss: 1.5378 - accuracy: 0.1446 - val_loss: 1.5473 - val_accuracy: 0.1456

Epoch 14/50

loss: 1.5357 - accuracy: 0.1449 - val_loss: 1.5310 - val_accuracy: 0.1457

Epoch 15/50

loss: 1.5424 - accuracy: 0.1446 - val_loss: 1.5269 - val_accuracy: 0.1461

Epoch 16/50

loss: 1.5314 - accuracy: 0.1450 - val_loss: 1.5392 - val_accuracy: 0.1456

Epoch 17/50

loss: 1.5309 - accuracy: 0.1451 - val_loss: 1.5567 - val_accuracy: 0.1454

Epoch 18/50

loss: 1.5279 - accuracy: 0.1450 - val_loss: 1.5561 - val_accuracy: 0.1452

Epoch 19/50

loss: 1.5311 - accuracy: 0.1450 - val_loss: 1.5400 - val_accuracy: 0.1460

Epoch 20/50

loss: 1.5332 - accuracy: 0.1449 - val_loss: 1.5347 - val_accuracy: 0.1460

Epoch 21/50

loss: 1.5319 - accuracy: 0.1452 - val_loss: 1.5410 - val_accuracy: 0.1458

Epoch 22/50

loss: 1.5327 - accuracy: 0.1449 - val_loss: 1.5352 - val_accuracy: 0.1460

Epoch 23/50

loss: 1.5278 - accuracy: 0.1451 - val_loss: 1.5289 - val_accuracy: 0.1458

Epoch 24/50

loss: 1.5234 - accuracy: 0.1451 - val_loss: 1.5568 - val_accuracy: 0.1449

Framework versions

  • Transformers 4.22.1
  • Torch 1.13.1
  • Datasets 2.5.1
  • Tokenizers 0.12.1
Downloads last month
18
Safetensors
Model size
178M params
Tensor type
I64
·
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for fimu-docproc-research/mbert-finetuned-pytorch

Finetuned
(551)
this model