--- license: apache-2.0 base_model: bert-base-multilingual-cased tags: - generated_from_trainer metrics: - accuracy model-index: - name: ms results: [] --- # ms This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.4335 - Accuracy: 0.7183 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - distributed_type: multi-GPU - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 50000 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:-----:|:-----:|:---------------:|:--------:| | 1.8607 | 0.05 | 500 | 1.6380 | 0.6876 | | 1.7966 | 0.1 | 1000 | 1.6063 | 0.6916 | | 1.7581 | 0.15 | 1500 | 1.5902 | 0.6949 | | 1.7313 | 0.2 | 2000 | 1.5726 | 0.6975 | | 1.7143 | 0.25 | 2500 | 1.5630 | 0.6992 | | 1.7074 | 0.3 | 3000 | 1.5562 | 0.6997 | | 1.699 | 0.34 | 3500 | 1.5440 | 0.7020 | | 1.6797 | 0.39 | 4000 | 1.5376 | 0.7026 | | 1.6813 | 0.44 | 4500 | 1.5355 | 0.7028 | | 1.6641 | 0.49 | 5000 | 1.5276 | 0.7041 | | 1.672 | 0.54 | 5500 | 1.5212 | 0.7048 | | 1.6506 | 0.59 | 6000 | 1.5167 | 0.7058 | | 1.6592 | 0.64 | 6500 | 1.5083 | 0.7069 | | 1.6386 | 0.69 | 7000 | 1.5050 | 0.7071 | | 1.6449 | 0.74 | 7500 | 1.5033 | 0.7078 | | 1.6362 | 0.79 | 8000 | 1.5032 | 0.7073 | | 1.6337 | 0.84 | 8500 | 1.4978 | 0.7082 | | 1.622 | 0.89 | 9000 | 1.4971 | 0.7089 | | 1.6279 | 0.94 | 9500 | 1.4919 | 0.7094 | | 1.6199 | 0.98 | 10000 | 1.4900 | 0.7096 | | 1.6218 | 1.03 | 10500 | 1.4914 | 0.7099 | | 1.6144 | 1.08 | 11000 | 1.4814 | 0.7113 | | 1.6017 | 1.13 | 11500 | 1.4789 | 0.7113 | | 1.6092 | 1.18 | 12000 | 1.4755 | 0.7119 | | 1.6083 | 1.23 | 12500 | 1.4766 | 0.7119 | | 1.6081 | 1.28 | 13000 | 1.4791 | 0.7113 | | 1.615 | 1.33 | 13500 | 1.4722 | 0.7127 | | 1.605 | 1.38 | 14000 | 1.4725 | 0.7124 | | 1.598 | 1.43 | 14500 | 1.4763 | 0.7119 | | 1.6004 | 1.48 | 15000 | 1.4661 | 0.7132 | | 1.6074 | 1.53 | 15500 | 1.4713 | 0.7123 | | 1.603 | 1.58 | 16000 | 1.4658 | 0.7135 | | 1.5928 | 1.62 | 16500 | 1.4646 | 0.7135 | | 1.5942 | 1.67 | 17000 | 1.4676 | 0.7132 | | 1.5914 | 1.72 | 17500 | 1.4604 | 0.7145 | | 1.5931 | 1.77 | 18000 | 1.4580 | 0.7142 | | 1.5808 | 1.82 | 18500 | 1.4606 | 0.7148 | | 1.5911 | 1.87 | 19000 | 1.4592 | 0.7143 | | 1.5842 | 1.92 | 19500 | 1.4622 | 0.7143 | | 1.5838 | 1.97 | 20000 | 1.4584 | 0.7146 | | 1.585 | 2.02 | 20500 | 1.4630 | 0.7139 | | 1.5772 | 2.07 | 21000 | 1.4557 | 0.7146 | | 1.589 | 2.12 | 21500 | 1.4556 | 0.7151 | | 1.5653 | 2.17 | 22000 | 1.4533 | 0.7154 | | 1.5753 | 2.22 | 22500 | 1.4574 | 0.7152 | | 1.5707 | 2.26 | 23000 | 1.4531 | 0.7155 | | 1.5744 | 2.31 | 23500 | 1.4534 | 0.7153 | | 1.5741 | 2.36 | 24000 | 1.4531 | 0.7155 | | 1.5696 | 2.41 | 24500 | 1.4480 | 0.7161 | | 1.5844 | 2.46 | 25000 | 1.4526 | 0.7157 | | 1.576 | 2.51 | 25500 | 1.4478 | 0.7160 | | 1.5621 | 2.56 | 26000 | 1.4497 | 0.7158 | | 1.5707 | 2.61 | 26500 | 1.4514 | 0.7159 | | 1.5819 | 2.66 | 27000 | 1.4478 | 0.7164 | | 1.5663 | 2.71 | 27500 | 1.4503 | 0.7159 | | 1.5834 | 2.76 | 28000 | 1.4507 | 0.7160 | | 1.5726 | 2.81 | 28500 | 1.4426 | 0.7165 | | 1.5695 | 2.86 | 29000 | 1.4466 | 0.7166 | | 1.5791 | 2.9 | 29500 | 1.4464 | 0.7167 | | 1.5711 | 2.95 | 30000 | 1.4434 | 0.7170 | | 1.5724 | 3.0 | 30500 | 1.4423 | 0.7169 | | 1.5648 | 3.05 | 31000 | 1.4410 | 0.7173 | | 1.5666 | 3.1 | 31500 | 1.4427 | 0.7168 | | 1.567 | 3.15 | 32000 | 1.4424 | 0.7170 | | 1.5569 | 3.2 | 32500 | 1.4441 | 0.7168 | | 1.5717 | 3.25 | 33000 | 1.4385 | 0.7182 | | 1.5585 | 3.3 | 33500 | 1.4416 | 0.7175 | | 1.5572 | 3.35 | 34000 | 1.4415 | 0.7174 | | 1.5585 | 3.4 | 34500 | 1.4368 | 0.7181 | | 1.5705 | 3.45 | 35000 | 1.4400 | 0.7178 | | 1.5712 | 3.5 | 35500 | 1.4420 | 0.7173 | | 1.5651 | 3.55 | 36000 | 1.4355 | 0.7186 | | 1.5595 | 3.59 | 36500 | 1.4363 | 0.7179 | | 1.5613 | 3.64 | 37000 | 1.4385 | 0.7179 | | 1.5594 | 3.69 | 37500 | 1.4422 | 0.7172 | | 1.5574 | 3.74 | 38000 | 1.4370 | 0.7179 | | 1.5557 | 3.79 | 38500 | 1.4410 | 0.7177 | | 1.5649 | 3.84 | 39000 | 1.4349 | 0.7181 | | 1.5635 | 3.89 | 39500 | 1.4406 | 0.7174 | | 1.5569 | 3.94 | 40000 | 1.4362 | 0.7182 | | 1.5661 | 3.99 | 40500 | 1.4369 | 0.7180 | | 1.5612 | 4.04 | 41000 | 1.4380 | 0.7178 | | 1.5632 | 4.09 | 41500 | 1.4374 | 0.7180 | | 1.5617 | 4.14 | 42000 | 1.4374 | 0.7178 | | 1.5452 | 4.19 | 42500 | 1.4341 | 0.7185 | | 1.5644 | 4.23 | 43000 | 1.4358 | 0.7181 | | 1.5448 | 4.28 | 43500 | 1.4392 | 0.7178 | | 1.559 | 4.33 | 44000 | 1.4363 | 0.7180 | | 1.5599 | 4.38 | 44500 | 1.4332 | 0.7185 | | 1.5586 | 4.43 | 45000 | 1.4391 | 0.7179 | | 1.5527 | 4.48 | 45500 | 1.4355 | 0.7183 | | 1.5592 | 4.53 | 46000 | 1.4314 | 0.7187 | | 1.55 | 4.58 | 46500 | 1.4352 | 0.7179 | | 1.5592 | 4.63 | 47000 | 1.4367 | 0.7180 | | 1.5586 | 4.68 | 47500 | 1.4346 | 0.7183 | | 1.5547 | 4.73 | 48000 | 1.4321 | 0.7192 | | 1.5572 | 4.78 | 48500 | 1.4397 | 0.7174 | | 1.5537 | 4.83 | 49000 | 1.4334 | 0.7186 | | 1.5546 | 4.87 | 49500 | 1.4347 | 0.7181 | | 1.5518 | 4.92 | 50000 | 1.4365 | 0.7183 | ### Framework versions - Transformers 4.36.2 - Pytorch 2.0.0 - Datasets 2.15.0 - Tokenizers 0.15.0