--- language: - id license: mit base_model: indolem/indobert-base-uncased tags: - generated_from_trainer model-index: - name: nerugm-seq_bn-2 results: [] --- # nerugm-seq_bn-2 This model is a fine-tuned version of [indolem/indobert-base-uncased](https://huggingface.co/indolem/indobert-base-uncased) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.1775 - Location Precision: 0.7821 - Location Recall: 0.8472 - Location F1: 0.8133 - Location Number: 72 - Organization Precision: 0.6701 - Organization Recall: 0.8667 - Organization F1: 0.7558 - Organization Number: 75 - Person Precision: 0.8654 - Person Recall: 0.9441 - Person F1: 0.9030 - Person Number: 143 - Quantity Precision: 0.5667 - Quantity Recall: 0.7391 - Quantity F1: 0.6415 - Quantity Number: 23 - Time Precision: 0.8148 - Time Recall: 0.8462 - Time F1: 0.8302 - Time Number: 26 - Overall Precision: 0.7732 - Overall Recall: 0.8850 - Overall F1: 0.8253 - Overall Accuracy: 0.9617 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 64 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 100.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Location Precision | Location Recall | Location F1 | Location Number | Organization Precision | Organization Recall | Organization F1 | Organization Number | Person Precision | Person Recall | Person F1 | Person Number | Quantity Precision | Quantity Recall | Quantity F1 | Quantity Number | Time Precision | Time Recall | Time F1 | Time Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:-----:|:-----:|:---------------:|:------------------:|:---------------:|:-----------:|:---------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------:|:-------------:|:---------:|:-------------:|:------------------:|:---------------:|:-----------:|:---------------:|:--------------:|:-----------:|:-------:|:-----------:|:-----------------:|:--------------:|:----------:|:----------------:| | 0.932 | 1.0 | 106 | 0.6092 | 0.0 | 0.0 | 0.0 | 72 | 0.0 | 0.0 | 0.0 | 75 | 0.0 | 0.0 | 0.0 | 143 | 0.0 | 0.0 | 0.0 | 23 | 0.0 | 0.0 | 0.0 | 26 | 0.0 | 0.0 | 0.0 | 0.8449 | | 0.5616 | 2.0 | 212 | 0.4137 | 0.55 | 0.1528 | 0.2391 | 72 | 0.1591 | 0.0933 | 0.1176 | 75 | 0.4046 | 0.4895 | 0.4430 | 143 | 0.0 | 0.0 | 0.0 | 23 | 0.5294 | 0.3462 | 0.4186 | 26 | 0.3804 | 0.2861 | 0.3266 | 0.8864 | | 0.4058 | 3.0 | 318 | 0.3135 | 0.6027 | 0.6111 | 0.6069 | 72 | 0.4412 | 0.4 | 0.4196 | 75 | 0.5540 | 0.8252 | 0.6629 | 143 | 0.12 | 0.1304 | 0.1250 | 23 | 0.7917 | 0.7308 | 0.76 | 26 | 0.5310 | 0.6313 | 0.5768 | 0.9209 | | 0.3002 | 4.0 | 424 | 0.2447 | 0.6180 | 0.7639 | 0.6832 | 72 | 0.5 | 0.5733 | 0.5342 | 75 | 0.6772 | 0.8951 | 0.7711 | 143 | 0.25 | 0.3478 | 0.2909 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.6066 | 0.7552 | 0.6728 | 0.9345 | | 0.2302 | 5.0 | 530 | 0.1956 | 0.6264 | 0.7917 | 0.6994 | 72 | 0.6 | 0.6 | 0.6 | 75 | 0.75 | 0.8811 | 0.8103 | 143 | 0.3077 | 0.5217 | 0.3871 | 23 | 0.84 | 0.8077 | 0.8235 | 26 | 0.6558 | 0.7699 | 0.7083 | 0.9410 | | 0.1943 | 6.0 | 636 | 0.1712 | 0.7059 | 0.8333 | 0.7643 | 72 | 0.5632 | 0.6533 | 0.6049 | 75 | 0.7840 | 0.8881 | 0.8328 | 143 | 0.3714 | 0.5652 | 0.4483 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.6826 | 0.7994 | 0.7364 | 0.9452 | | 0.1777 | 7.0 | 742 | 0.1684 | 0.7317 | 0.8333 | 0.7792 | 72 | 0.5895 | 0.7467 | 0.6588 | 75 | 0.7840 | 0.8881 | 0.8328 | 143 | 0.3947 | 0.6522 | 0.4918 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.6897 | 0.8260 | 0.7517 | 0.9467 | | 0.162 | 8.0 | 848 | 0.1578 | 0.6897 | 0.8333 | 0.7547 | 72 | 0.6173 | 0.6667 | 0.6410 | 75 | 0.8165 | 0.9021 | 0.8571 | 143 | 0.4167 | 0.6522 | 0.5085 | 23 | 0.7097 | 0.8462 | 0.7719 | 26 | 0.7023 | 0.8142 | 0.7541 | 0.9492 | | 0.154 | 9.0 | 954 | 0.1556 | 0.7093 | 0.8472 | 0.7722 | 72 | 0.625 | 0.7333 | 0.6748 | 75 | 0.8176 | 0.9091 | 0.8609 | 143 | 0.4706 | 0.6957 | 0.5614 | 23 | 0.7419 | 0.8846 | 0.8070 | 26 | 0.7161 | 0.8407 | 0.7734 | 0.9522 | | 0.1445 | 10.0 | 1060 | 0.1476 | 0.7349 | 0.8472 | 0.7871 | 72 | 0.6429 | 0.72 | 0.6792 | 75 | 0.8187 | 0.9161 | 0.8647 | 143 | 0.4722 | 0.7391 | 0.5763 | 23 | 0.6875 | 0.8462 | 0.7586 | 26 | 0.7215 | 0.8407 | 0.7766 | 0.9530 | | 0.1367 | 11.0 | 1166 | 0.1449 | 0.7209 | 0.8611 | 0.7848 | 72 | 0.6512 | 0.7467 | 0.6957 | 75 | 0.8397 | 0.9161 | 0.8763 | 143 | 0.4857 | 0.7391 | 0.5862 | 23 | 0.6875 | 0.8462 | 0.7586 | 26 | 0.7291 | 0.8496 | 0.7847 | 0.9545 | | 0.132 | 12.0 | 1272 | 0.1459 | 0.7468 | 0.8194 | 0.7815 | 72 | 0.6420 | 0.6933 | 0.6667 | 75 | 0.8228 | 0.9091 | 0.8638 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.7667 | 0.8846 | 0.8214 | 26 | 0.7375 | 0.8289 | 0.7806 | 0.9522 | | 0.1276 | 13.0 | 1378 | 0.1365 | 0.7375 | 0.8194 | 0.7763 | 72 | 0.6782 | 0.7867 | 0.7284 | 75 | 0.8397 | 0.9161 | 0.8763 | 143 | 0.5862 | 0.7391 | 0.6538 | 23 | 0.8214 | 0.8846 | 0.8519 | 26 | 0.7605 | 0.8525 | 0.8039 | 0.9572 | | 0.1248 | 14.0 | 1484 | 0.1440 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6667 | 0.8533 | 0.7485 | 75 | 0.8397 | 0.9161 | 0.8763 | 143 | 0.5312 | 0.7391 | 0.6182 | 23 | 0.7333 | 0.8462 | 0.7857 | 26 | 0.7506 | 0.8702 | 0.8060 | 0.9562 | | 0.1209 | 15.0 | 1590 | 0.1367 | 0.7662 | 0.8194 | 0.7919 | 72 | 0.6739 | 0.8267 | 0.7425 | 75 | 0.8533 | 0.8951 | 0.8737 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.7639 | 0.8496 | 0.8045 | 0.9577 | | 0.1172 | 16.0 | 1696 | 0.1445 | 0.7349 | 0.8472 | 0.7871 | 72 | 0.7033 | 0.8533 | 0.7711 | 75 | 0.8408 | 0.9231 | 0.8800 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7532 | 0.8732 | 0.8087 | 0.9562 | | 0.1115 | 17.0 | 1802 | 0.1387 | 0.7692 | 0.8333 | 0.8 | 72 | 0.7333 | 0.88 | 0.8 | 75 | 0.8462 | 0.9231 | 0.8829 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.7694 | 0.8761 | 0.8193 | 0.9580 | | 0.1103 | 18.0 | 1908 | 0.1388 | 0.7470 | 0.8611 | 0.8 | 72 | 0.7326 | 0.84 | 0.7826 | 75 | 0.8562 | 0.9161 | 0.8851 | 143 | 0.5294 | 0.7826 | 0.6316 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7688 | 0.8732 | 0.8177 | 0.9590 | | 0.1071 | 19.0 | 2014 | 0.1341 | 0.7662 | 0.8194 | 0.7919 | 72 | 0.7442 | 0.8533 | 0.7950 | 75 | 0.8481 | 0.9371 | 0.8904 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7708 | 0.8732 | 0.8188 | 0.9595 | | 0.1028 | 20.0 | 2120 | 0.1560 | 0.6932 | 0.8472 | 0.7625 | 72 | 0.6538 | 0.9067 | 0.7598 | 75 | 0.8428 | 0.9371 | 0.8874 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.6364 | 0.8077 | 0.7119 | 26 | 0.7218 | 0.8879 | 0.7963 | 0.9520 | | 0.1016 | 21.0 | 2226 | 0.1470 | 0.7294 | 0.8611 | 0.7898 | 72 | 0.7 | 0.9333 | 0.8 | 75 | 0.8438 | 0.9441 | 0.8911 | 143 | 0.5455 | 0.7826 | 0.6429 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7543 | 0.9056 | 0.8231 | 0.9567 | | 0.0999 | 22.0 | 2332 | 0.1354 | 0.7564 | 0.8194 | 0.7867 | 72 | 0.7312 | 0.9067 | 0.8095 | 75 | 0.8526 | 0.9301 | 0.8896 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7686 | 0.8820 | 0.8214 | 0.9597 | | 0.0953 | 23.0 | 2438 | 0.1358 | 0.7867 | 0.8194 | 0.8027 | 72 | 0.7340 | 0.92 | 0.8166 | 75 | 0.8516 | 0.9231 | 0.8859 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7726 | 0.8820 | 0.8237 | 0.9597 | | 0.0932 | 24.0 | 2544 | 0.1410 | 0.7531 | 0.8472 | 0.7974 | 72 | 0.7204 | 0.8933 | 0.7976 | 75 | 0.8526 | 0.9301 | 0.8896 | 143 | 0.4571 | 0.6957 | 0.5517 | 23 | 0.7333 | 0.8462 | 0.7857 | 26 | 0.7570 | 0.8820 | 0.8147 | 0.9575 | | 0.0914 | 25.0 | 2650 | 0.1396 | 0.7531 | 0.8472 | 0.7974 | 72 | 0.7083 | 0.9067 | 0.7953 | 75 | 0.8516 | 0.9231 | 0.8859 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7595 | 0.8850 | 0.8174 | 0.9585 | | 0.0898 | 26.0 | 2756 | 0.1374 | 0.7867 | 0.8194 | 0.8027 | 72 | 0.7363 | 0.8933 | 0.8072 | 75 | 0.8471 | 0.9301 | 0.8867 | 143 | 0.5161 | 0.6957 | 0.5926 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7749 | 0.8732 | 0.8211 | 0.9595 | | 0.0862 | 27.0 | 2862 | 0.1478 | 0.7229 | 0.8333 | 0.7742 | 72 | 0.7222 | 0.8667 | 0.7879 | 75 | 0.8428 | 0.9371 | 0.8874 | 143 | 0.5 | 0.7826 | 0.6102 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7531 | 0.8820 | 0.8125 | 0.9580 | | 0.0847 | 28.0 | 2968 | 0.1362 | 0.75 | 0.8333 | 0.7895 | 72 | 0.7333 | 0.88 | 0.8 | 75 | 0.8516 | 0.9231 | 0.8859 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7708 | 0.8732 | 0.8188 | 0.9600 | | 0.0837 | 29.0 | 3074 | 0.1457 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6731 | 0.9333 | 0.7821 | 75 | 0.8506 | 0.9161 | 0.8822 | 143 | 0.5312 | 0.7391 | 0.6182 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.7563 | 0.8879 | 0.8168 | 0.9572 | | 0.0823 | 30.0 | 3180 | 0.1313 | 0.7763 | 0.8194 | 0.7973 | 72 | 0.7253 | 0.88 | 0.7952 | 75 | 0.8627 | 0.9231 | 0.8919 | 143 | 0.5 | 0.6522 | 0.5660 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.7778 | 0.8673 | 0.8201 | 0.9617 | | 0.0818 | 31.0 | 3286 | 0.1331 | 0.7532 | 0.8056 | 0.7785 | 72 | 0.7262 | 0.8133 | 0.7673 | 75 | 0.8526 | 0.9301 | 0.8896 | 143 | 0.5161 | 0.6957 | 0.5926 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7707 | 0.8525 | 0.8095 | 0.9600 | | 0.0791 | 32.0 | 3392 | 0.1391 | 0.7317 | 0.8333 | 0.7792 | 72 | 0.7204 | 0.8933 | 0.7976 | 75 | 0.8627 | 0.9231 | 0.8919 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7615 | 0.8761 | 0.8148 | 0.9607 | | 0.0767 | 33.0 | 3498 | 0.1348 | 0.7662 | 0.8194 | 0.7919 | 72 | 0.7204 | 0.8933 | 0.7976 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.4839 | 0.6522 | 0.5556 | 23 | 0.7333 | 0.8462 | 0.7857 | 26 | 0.7700 | 0.8791 | 0.8209 | 0.9612 | | 0.0747 | 34.0 | 3604 | 0.1479 | 0.7317 | 0.8333 | 0.7792 | 72 | 0.6939 | 0.9067 | 0.7861 | 75 | 0.8428 | 0.9371 | 0.8874 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.7333 | 0.8462 | 0.7857 | 26 | 0.7469 | 0.8879 | 0.8113 | 0.9570 | | 0.0748 | 35.0 | 3710 | 0.1600 | 0.7262 | 0.8472 | 0.7821 | 72 | 0.5948 | 0.92 | 0.7225 | 75 | 0.875 | 0.9301 | 0.9017 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7277 | 0.8909 | 0.8011 | 0.9525 | | 0.0748 | 36.0 | 3816 | 0.1407 | 0.7407 | 0.8333 | 0.7843 | 72 | 0.7053 | 0.8933 | 0.7882 | 75 | 0.8535 | 0.9371 | 0.8933 | 143 | 0.5172 | 0.6522 | 0.5769 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.7641 | 0.8791 | 0.8176 | 0.9600 | | 0.0694 | 37.0 | 3922 | 0.1407 | 0.76 | 0.7917 | 0.7755 | 72 | 0.6832 | 0.92 | 0.7841 | 75 | 0.8581 | 0.9301 | 0.8926 | 143 | 0.5357 | 0.6522 | 0.5882 | 23 | 0.7586 | 0.8462 | 0.8 | 26 | 0.7629 | 0.8732 | 0.8143 | 0.9600 | | 0.0674 | 38.0 | 4028 | 0.1403 | 0.7632 | 0.8056 | 0.7838 | 72 | 0.7128 | 0.8933 | 0.7929 | 75 | 0.8590 | 0.9371 | 0.8963 | 143 | 0.5161 | 0.6957 | 0.5926 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7688 | 0.8732 | 0.8177 | 0.9597 | | 0.0677 | 39.0 | 4134 | 0.1457 | 0.7632 | 0.8056 | 0.7838 | 72 | 0.7041 | 0.92 | 0.7977 | 75 | 0.8581 | 0.9301 | 0.8926 | 143 | 0.5312 | 0.7391 | 0.6182 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.7686 | 0.8820 | 0.8214 | 0.9590 | | 0.0667 | 40.0 | 4240 | 0.1403 | 0.7532 | 0.8056 | 0.7785 | 72 | 0.6957 | 0.8533 | 0.7665 | 75 | 0.8636 | 0.9301 | 0.8956 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7664 | 0.8614 | 0.8111 | 0.9607 | | 0.0647 | 41.0 | 4346 | 0.1435 | 0.7375 | 0.8194 | 0.7763 | 72 | 0.6735 | 0.88 | 0.7630 | 75 | 0.8599 | 0.9441 | 0.9 | 143 | 0.5357 | 0.6522 | 0.5882 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7570 | 0.8732 | 0.8110 | 0.9602 | | 0.0625 | 42.0 | 4452 | 0.1474 | 0.7468 | 0.8194 | 0.7815 | 72 | 0.6837 | 0.8933 | 0.7746 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5 | 0.6957 | 0.5818 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7583 | 0.8791 | 0.8142 | 0.9592 | | 0.0639 | 43.0 | 4558 | 0.1503 | 0.7407 | 0.8333 | 0.7843 | 72 | 0.7065 | 0.8667 | 0.7784 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7641 | 0.8791 | 0.8176 | 0.9607 | | 0.0615 | 44.0 | 4664 | 0.1499 | 0.7439 | 0.8472 | 0.7922 | 72 | 0.6970 | 0.92 | 0.7931 | 75 | 0.8710 | 0.9441 | 0.9060 | 143 | 0.4839 | 0.6522 | 0.5556 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7659 | 0.8879 | 0.8224 | 0.9612 | | 0.0595 | 45.0 | 4770 | 0.1458 | 0.7632 | 0.8056 | 0.7838 | 72 | 0.6837 | 0.8933 | 0.7746 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.4839 | 0.6522 | 0.5556 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7642 | 0.8702 | 0.8138 | 0.9600 | | 0.0608 | 46.0 | 4876 | 0.1408 | 0.7564 | 0.8194 | 0.7867 | 72 | 0.7191 | 0.8533 | 0.7805 | 75 | 0.8701 | 0.9371 | 0.9024 | 143 | 0.5357 | 0.6522 | 0.5882 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7813 | 0.8643 | 0.8207 | 0.9622 | | 0.0612 | 47.0 | 4982 | 0.1553 | 0.7532 | 0.8056 | 0.7785 | 72 | 0.69 | 0.92 | 0.7886 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.4848 | 0.6957 | 0.5714 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7608 | 0.8820 | 0.8169 | 0.9590 | | 0.0602 | 48.0 | 5088 | 0.1468 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.7097 | 0.88 | 0.7857 | 75 | 0.8701 | 0.9371 | 0.9024 | 143 | 0.4839 | 0.6522 | 0.5556 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7694 | 0.8761 | 0.8193 | 0.9617 | | 0.0581 | 49.0 | 5194 | 0.1489 | 0.75 | 0.8333 | 0.7895 | 72 | 0.6970 | 0.92 | 0.7931 | 75 | 0.8599 | 0.9441 | 0.9 | 143 | 0.5172 | 0.6522 | 0.5769 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7673 | 0.8850 | 0.8219 | 0.9615 | | 0.0553 | 50.0 | 5300 | 0.1541 | 0.7662 | 0.8194 | 0.7919 | 72 | 0.6735 | 0.88 | 0.7630 | 75 | 0.8701 | 0.9371 | 0.9024 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7655 | 0.8761 | 0.8171 | 0.9595 | | 0.0555 | 51.0 | 5406 | 0.1573 | 0.7792 | 0.8333 | 0.8054 | 72 | 0.6571 | 0.92 | 0.7667 | 75 | 0.8544 | 0.9441 | 0.8970 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7652 | 0.8938 | 0.8245 | 0.9590 | | 0.0562 | 52.0 | 5512 | 0.1513 | 0.7468 | 0.8194 | 0.7815 | 72 | 0.6979 | 0.8933 | 0.7836 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5484 | 0.7391 | 0.6296 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7661 | 0.8791 | 0.8187 | 0.9602 | | 0.0556 | 53.0 | 5618 | 0.1523 | 0.7564 | 0.8194 | 0.7867 | 72 | 0.7010 | 0.9067 | 0.7907 | 75 | 0.8544 | 0.9441 | 0.8970 | 143 | 0.5312 | 0.7391 | 0.6182 | 23 | 0.8 | 0.7692 | 0.7843 | 26 | 0.7667 | 0.8820 | 0.8203 | 0.9605 | | 0.0549 | 54.0 | 5724 | 0.1687 | 0.7595 | 0.8333 | 0.7947 | 72 | 0.6339 | 0.9467 | 0.7594 | 75 | 0.8526 | 0.9301 | 0.8896 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7402 | 0.8909 | 0.8086 | 0.9555 | | 0.0533 | 55.0 | 5830 | 0.1602 | 0.75 | 0.8333 | 0.7895 | 72 | 0.6574 | 0.9467 | 0.7760 | 75 | 0.8590 | 0.9371 | 0.8963 | 143 | 0.5484 | 0.7391 | 0.6296 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7537 | 0.8938 | 0.8178 | 0.9590 | | 0.0511 | 56.0 | 5936 | 0.1552 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6875 | 0.88 | 0.7719 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5 | 0.7391 | 0.5965 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7608 | 0.8820 | 0.8169 | 0.9605 | | 0.0497 | 57.0 | 6042 | 0.1536 | 0.7692 | 0.8333 | 0.8 | 72 | 0.6907 | 0.8933 | 0.7791 | 75 | 0.8758 | 0.9371 | 0.9054 | 143 | 0.4848 | 0.6957 | 0.5714 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7680 | 0.8791 | 0.8198 | 0.9612 | | 0.0492 | 58.0 | 6148 | 0.1586 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6768 | 0.8933 | 0.7701 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7614 | 0.8850 | 0.8186 | 0.9600 | | 0.0489 | 59.0 | 6254 | 0.1573 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6869 | 0.9067 | 0.7816 | 75 | 0.8701 | 0.9371 | 0.9024 | 143 | 0.4848 | 0.6957 | 0.5714 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7634 | 0.8850 | 0.8197 | 0.9607 | | 0.0494 | 60.0 | 6360 | 0.1614 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6970 | 0.92 | 0.7931 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.4848 | 0.6957 | 0.5714 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7659 | 0.8879 | 0.8224 | 0.9605 | | 0.0476 | 61.0 | 6466 | 0.1569 | 0.7564 | 0.8194 | 0.7867 | 72 | 0.6875 | 0.88 | 0.7719 | 75 | 0.8535 | 0.9371 | 0.8933 | 143 | 0.5152 | 0.7391 | 0.6071 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7621 | 0.8791 | 0.8164 | 0.9610 | | 0.0469 | 62.0 | 6572 | 0.1600 | 0.7531 | 0.8472 | 0.7974 | 72 | 0.7021 | 0.88 | 0.7811 | 75 | 0.8535 | 0.9371 | 0.8933 | 143 | 0.5 | 0.6957 | 0.5818 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7621 | 0.8791 | 0.8164 | 0.9612 | | 0.0482 | 63.0 | 6678 | 0.1566 | 0.775 | 0.8611 | 0.8158 | 72 | 0.6837 | 0.8933 | 0.7746 | 75 | 0.875 | 0.9301 | 0.9017 | 143 | 0.4839 | 0.6522 | 0.5556 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7706 | 0.8820 | 0.8226 | 0.9622 | | 0.0481 | 64.0 | 6784 | 0.1712 | 0.75 | 0.8333 | 0.7895 | 72 | 0.6604 | 0.9333 | 0.7735 | 75 | 0.8544 | 0.9441 | 0.8970 | 143 | 0.4848 | 0.6957 | 0.5714 | 23 | 0.7857 | 0.8462 | 0.8148 | 26 | 0.7481 | 0.8938 | 0.8145 | 0.9590 | | 0.0437 | 65.0 | 6890 | 0.1590 | 0.7595 | 0.8333 | 0.7947 | 72 | 0.6701 | 0.8667 | 0.7558 | 75 | 0.8710 | 0.9441 | 0.9060 | 143 | 0.5172 | 0.6522 | 0.5769 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7649 | 0.8732 | 0.8154 | 0.9617 | | 0.0434 | 66.0 | 6996 | 0.1602 | 0.7595 | 0.8333 | 0.7947 | 72 | 0.6735 | 0.88 | 0.7630 | 75 | 0.8766 | 0.9441 | 0.9091 | 143 | 0.5517 | 0.6957 | 0.6154 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7746 | 0.8820 | 0.8248 | 0.9630 | | 0.0431 | 67.0 | 7102 | 0.1639 | 0.7595 | 0.8333 | 0.7947 | 72 | 0.6907 | 0.8933 | 0.7791 | 75 | 0.8710 | 0.9441 | 0.9060 | 143 | 0.5312 | 0.7391 | 0.6182 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7718 | 0.8879 | 0.8258 | 0.9620 | | 0.0428 | 68.0 | 7208 | 0.1702 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6733 | 0.9067 | 0.7727 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5484 | 0.7391 | 0.6296 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7665 | 0.8909 | 0.8240 | 0.9610 | | 0.0422 | 69.0 | 7314 | 0.1660 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6869 | 0.9067 | 0.7816 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7744 | 0.8909 | 0.8285 | 0.9627 | | 0.0443 | 70.0 | 7420 | 0.1704 | 0.7531 | 0.8472 | 0.7974 | 72 | 0.6733 | 0.9067 | 0.7727 | 75 | 0.8590 | 0.9371 | 0.8963 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7614 | 0.8850 | 0.8186 | 0.9607 | | 0.0422 | 71.0 | 7526 | 0.1673 | 0.7531 | 0.8472 | 0.7974 | 72 | 0.6979 | 0.8933 | 0.7836 | 75 | 0.8710 | 0.9441 | 0.9060 | 143 | 0.5484 | 0.7391 | 0.6296 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7738 | 0.8879 | 0.8269 | 0.9622 | | 0.0429 | 72.0 | 7632 | 0.1664 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6979 | 0.8933 | 0.7836 | 75 | 0.8710 | 0.9441 | 0.9060 | 143 | 0.5 | 0.6957 | 0.5818 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7732 | 0.8850 | 0.8253 | 0.9622 | | 0.0433 | 73.0 | 7738 | 0.1627 | 0.7662 | 0.8194 | 0.7919 | 72 | 0.6837 | 0.8933 | 0.7746 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5517 | 0.6957 | 0.6154 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7740 | 0.8791 | 0.8232 | 0.9625 | | 0.0403 | 74.0 | 7844 | 0.1684 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6804 | 0.88 | 0.7674 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7706 | 0.8820 | 0.8226 | 0.9615 | | 0.0415 | 75.0 | 7950 | 0.1694 | 0.7375 | 0.8194 | 0.7763 | 72 | 0.6634 | 0.8933 | 0.7614 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8077 | 0.8077 | 0.8077 | 26 | 0.7577 | 0.8761 | 0.8126 | 0.9602 | | 0.0405 | 76.0 | 8056 | 0.1741 | 0.7531 | 0.8472 | 0.7974 | 72 | 0.6505 | 0.8933 | 0.7528 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7551 | 0.8820 | 0.8136 | 0.9597 | | 0.0399 | 77.0 | 8162 | 0.1697 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.67 | 0.8933 | 0.7657 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7608 | 0.8820 | 0.8169 | 0.9605 | | 0.0386 | 78.0 | 8268 | 0.1652 | 0.7922 | 0.8472 | 0.8188 | 72 | 0.6813 | 0.8267 | 0.7470 | 75 | 0.8766 | 0.9441 | 0.9091 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7831 | 0.8732 | 0.8257 | 0.9630 | | 0.0373 | 79.0 | 8374 | 0.1729 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6635 | 0.92 | 0.7709 | 75 | 0.8710 | 0.9441 | 0.9060 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7690 | 0.8938 | 0.8267 | 0.9612 | | 0.0391 | 80.0 | 8480 | 0.1742 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6735 | 0.88 | 0.7630 | 75 | 0.8645 | 0.9371 | 0.8993 | 143 | 0.5484 | 0.7391 | 0.6296 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7628 | 0.8820 | 0.8181 | 0.9605 | | 0.0375 | 81.0 | 8586 | 0.1745 | 0.7595 | 0.8333 | 0.7947 | 72 | 0.6804 | 0.88 | 0.7674 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7667 | 0.8820 | 0.8203 | 0.9612 | | 0.0388 | 82.0 | 8692 | 0.1736 | 0.7595 | 0.8333 | 0.7947 | 72 | 0.6804 | 0.88 | 0.7674 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7667 | 0.8820 | 0.8203 | 0.9612 | | 0.0378 | 83.0 | 8798 | 0.1751 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.6804 | 0.88 | 0.7674 | 75 | 0.8599 | 0.9441 | 0.9 | 143 | 0.5484 | 0.7391 | 0.6296 | 23 | 0.75 | 0.8077 | 0.7778 | 26 | 0.7634 | 0.8850 | 0.8197 | 0.9610 | | 0.0381 | 84.0 | 8904 | 0.1737 | 0.7662 | 0.8194 | 0.7919 | 72 | 0.69 | 0.92 | 0.7886 | 75 | 0.8710 | 0.9441 | 0.9060 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7784 | 0.8909 | 0.8308 | 0.9620 | | 0.0363 | 85.0 | 9010 | 0.1721 | 0.7595 | 0.8333 | 0.7947 | 72 | 0.6809 | 0.8533 | 0.7574 | 75 | 0.8491 | 0.9441 | 0.8940 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7655 | 0.8761 | 0.8171 | 0.9612 | | 0.0403 | 86.0 | 9116 | 0.1772 | 0.7625 | 0.8472 | 0.8026 | 72 | 0.69 | 0.92 | 0.7886 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7710 | 0.8938 | 0.8279 | 0.9615 | | 0.0365 | 87.0 | 9222 | 0.1748 | 0.7692 | 0.8333 | 0.8 | 72 | 0.6667 | 0.88 | 0.7586 | 75 | 0.8599 | 0.9441 | 0.9 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7621 | 0.8791 | 0.8164 | 0.9602 | | 0.035 | 88.0 | 9328 | 0.1746 | 0.7692 | 0.8333 | 0.8 | 72 | 0.6701 | 0.8667 | 0.7558 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7680 | 0.8791 | 0.8198 | 0.9610 | | 0.0355 | 89.0 | 9434 | 0.1779 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6735 | 0.88 | 0.7630 | 75 | 0.8544 | 0.9441 | 0.8970 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7653 | 0.8850 | 0.8208 | 0.9607 | | 0.0371 | 90.0 | 9540 | 0.1754 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6809 | 0.8533 | 0.7574 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7746 | 0.8820 | 0.8248 | 0.9615 | | 0.0362 | 91.0 | 9646 | 0.1716 | 0.7821 | 0.8472 | 0.8133 | 72 | 0.6957 | 0.8533 | 0.7665 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7801 | 0.8791 | 0.8266 | 0.9625 | | 0.0368 | 92.0 | 9752 | 0.1745 | 0.7692 | 0.8333 | 0.8 | 72 | 0.6598 | 0.8533 | 0.7442 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7655 | 0.8761 | 0.8171 | 0.9612 | | 0.036 | 93.0 | 9858 | 0.1764 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6735 | 0.88 | 0.7630 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7667 | 0.8820 | 0.8203 | 0.9610 | | 0.0367 | 94.0 | 9964 | 0.1744 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6771 | 0.8667 | 0.7602 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7706 | 0.8820 | 0.8226 | 0.9612 | | 0.0346 | 95.0 | 10070 | 0.1790 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6735 | 0.88 | 0.7630 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7692 | 0.8850 | 0.8230 | 0.9612 | | 0.0365 | 96.0 | 10176 | 0.1783 | 0.7821 | 0.8472 | 0.8133 | 72 | 0.6768 | 0.8933 | 0.7701 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7718 | 0.8879 | 0.8258 | 0.9617 | | 0.0341 | 97.0 | 10282 | 0.1792 | 0.7722 | 0.8472 | 0.8079 | 72 | 0.6768 | 0.8933 | 0.7701 | 75 | 0.8599 | 0.9441 | 0.9 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.7778 | 0.8077 | 0.7925 | 26 | 0.7679 | 0.8879 | 0.8235 | 0.9610 | | 0.0355 | 98.0 | 10388 | 0.1753 | 0.7821 | 0.8472 | 0.8133 | 72 | 0.6915 | 0.8667 | 0.7692 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5333 | 0.6957 | 0.6038 | 23 | 0.8462 | 0.8462 | 0.8462 | 26 | 0.7786 | 0.8820 | 0.8271 | 0.9620 | | 0.0369 | 99.0 | 10494 | 0.1775 | 0.7821 | 0.8472 | 0.8133 | 72 | 0.6701 | 0.8667 | 0.7558 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7732 | 0.8850 | 0.8253 | 0.9617 | | 0.0366 | 100.0 | 10600 | 0.1775 | 0.7821 | 0.8472 | 0.8133 | 72 | 0.6701 | 0.8667 | 0.7558 | 75 | 0.8654 | 0.9441 | 0.9030 | 143 | 0.5667 | 0.7391 | 0.6415 | 23 | 0.8148 | 0.8462 | 0.8302 | 26 | 0.7732 | 0.8850 | 0.8253 | 0.9617 | ### Framework versions - Transformers 4.40.2 - Pytorch 2.3.0+cu121 - Datasets 2.19.1 - Tokenizers 0.19.1