--- base_model: vinai/phobert-base-v2 tags: - generated_from_trainer metrics: - accuracy - f1 model-index: - name: PhoBert_Lexical_Dataset59KCoDuoi results: [] --- # PhoBert_Lexical_Dataset59KCoDuoi This model is a fine-tuned version of [vinai/phobert-base-v2](https://huggingface.co/vinai/phobert-base-v2) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.2741 - Accuracy: 0.9600 - F1: 0.9602 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 20 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | |:-------------:|:-------:|:-----:|:---------------:|:--------:|:------:| | No log | 0.2558 | 200 | 0.1794 | 0.9396 | 0.9400 | | No log | 0.5115 | 400 | 0.1533 | 0.9475 | 0.9479 | | No log | 0.7673 | 600 | 0.1522 | 0.9496 | 0.9499 | | 0.1767 | 1.0230 | 800 | 0.1494 | 0.9542 | 0.9545 | | 0.1767 | 1.2788 | 1000 | 0.1485 | 0.9519 | 0.9520 | | 0.1767 | 1.5345 | 1200 | 0.1608 | 0.9524 | 0.9523 | | 0.1767 | 1.7903 | 1400 | 0.1223 | 0.9580 | 0.9582 | | 0.1176 | 2.0460 | 1600 | 0.1462 | 0.9600 | 0.9603 | | 0.1176 | 2.3018 | 1800 | 0.1363 | 0.9588 | 0.9591 | | 0.1176 | 2.5575 | 2000 | 0.1441 | 0.9574 | 0.9577 | | 0.1176 | 2.8133 | 2200 | 0.1369 | 0.9566 | 0.9568 | | 0.0972 | 3.0691 | 2400 | 0.1530 | 0.9547 | 0.9550 | | 0.0972 | 3.3248 | 2600 | 0.1278 | 0.9607 | 0.9608 | | 0.0972 | 3.5806 | 2800 | 0.1334 | 0.9604 | 0.9606 | | 0.0972 | 3.8363 | 3000 | 0.1280 | 0.9608 | 0.9609 | | 0.0821 | 4.0921 | 3200 | 0.1379 | 0.9603 | 0.9604 | | 0.0821 | 4.3478 | 3400 | 0.1466 | 0.9587 | 0.9589 | | 0.0821 | 4.6036 | 3600 | 0.1379 | 0.9604 | 0.9606 | | 0.0821 | 4.8593 | 3800 | 0.1347 | 0.9606 | 0.9607 | | 0.0687 | 5.1151 | 4000 | 0.1492 | 0.9614 | 0.9614 | | 0.0687 | 5.3708 | 4200 | 0.1611 | 0.9606 | 0.9606 | | 0.0687 | 5.6266 | 4400 | 0.1407 | 0.9594 | 0.9596 | | 0.0687 | 5.8824 | 4600 | 0.1446 | 0.9590 | 0.9591 | | 0.0584 | 6.1381 | 4800 | 0.1659 | 0.9575 | 0.9578 | | 0.0584 | 6.3939 | 5000 | 0.1666 | 0.9602 | 0.9602 | | 0.0584 | 6.6496 | 5200 | 0.1683 | 0.9586 | 0.9588 | | 0.0584 | 6.9054 | 5400 | 0.1668 | 0.9609 | 0.9611 | | 0.0477 | 7.1611 | 5600 | 0.1844 | 0.9580 | 0.9582 | | 0.0477 | 7.4169 | 5800 | 0.1695 | 0.9626 | 0.9627 | | 0.0477 | 7.6726 | 6000 | 0.1767 | 0.9596 | 0.9597 | | 0.0477 | 7.9284 | 6200 | 0.1960 | 0.9594 | 0.9596 | | 0.0397 | 8.1841 | 6400 | 0.1932 | 0.9599 | 0.9600 | | 0.0397 | 8.4399 | 6600 | 0.1990 | 0.9593 | 0.9594 | | 0.0397 | 8.6957 | 6800 | 0.1999 | 0.9602 | 0.9603 | | 0.0397 | 8.9514 | 7000 | 0.1803 | 0.9577 | 0.9580 | | 0.0349 | 9.2072 | 7200 | 0.2082 | 0.9574 | 0.9575 | | 0.0349 | 9.4629 | 7400 | 0.2075 | 0.9597 | 0.9598 | | 0.0349 | 9.7187 | 7600 | 0.2269 | 0.9577 | 0.9577 | | 0.0349 | 9.9744 | 7800 | 0.1990 | 0.9602 | 0.9602 | | 0.0294 | 10.2302 | 8000 | 0.1987 | 0.9599 | 0.9600 | | 0.0294 | 10.4859 | 8200 | 0.2066 | 0.9563 | 0.9563 | | 0.0294 | 10.7417 | 8400 | 0.2149 | 0.9595 | 0.9597 | | 0.0257 | 10.9974 | 8600 | 0.2179 | 0.9609 | 0.9610 | | 0.0257 | 11.2532 | 8800 | 0.2337 | 0.9593 | 0.9594 | | 0.0257 | 11.5090 | 9000 | 0.2499 | 0.9573 | 0.9573 | | 0.0257 | 11.7647 | 9200 | 0.2323 | 0.9575 | 0.9577 | | 0.021 | 12.0205 | 9400 | 0.2330 | 0.9599 | 0.9601 | | 0.021 | 12.2762 | 9600 | 0.2321 | 0.9603 | 0.9604 | | 0.021 | 12.5320 | 9800 | 0.2431 | 0.9594 | 0.9594 | | 0.021 | 12.7877 | 10000 | 0.2487 | 0.9581 | 0.9583 | | 0.017 | 13.0435 | 10200 | 0.2606 | 0.9570 | 0.9570 | | 0.017 | 13.2992 | 10400 | 0.2450 | 0.9582 | 0.9583 | | 0.017 | 13.5550 | 10600 | 0.2647 | 0.9593 | 0.9596 | | 0.017 | 13.8107 | 10800 | 0.2494 | 0.9595 | 0.9597 | | 0.0155 | 14.0665 | 11000 | 0.2482 | 0.9582 | 0.9584 | | 0.0155 | 14.3223 | 11200 | 0.2552 | 0.9605 | 0.9606 | | 0.0155 | 14.5780 | 11400 | 0.2581 | 0.9583 | 0.9585 | | 0.0155 | 14.8338 | 11600 | 0.2553 | 0.9609 | 0.9611 | | 0.0146 | 15.0895 | 11800 | 0.2601 | 0.9591 | 0.9592 | | 0.0146 | 15.3453 | 12000 | 0.2574 | 0.9593 | 0.9594 | | 0.0146 | 15.6010 | 12200 | 0.2562 | 0.9614 | 0.9615 | | 0.0146 | 15.8568 | 12400 | 0.2588 | 0.9596 | 0.9597 | | 0.0114 | 16.1125 | 12600 | 0.2621 | 0.9581 | 0.9581 | | 0.0114 | 16.3683 | 12800 | 0.2593 | 0.9591 | 0.9593 | | 0.0114 | 16.6240 | 13000 | 0.2611 | 0.9607 | 0.9608 | | 0.0114 | 16.8798 | 13200 | 0.2668 | 0.9600 | 0.9602 | | 0.0091 | 17.1355 | 13400 | 0.2554 | 0.9618 | 0.9620 | | 0.0091 | 17.3913 | 13600 | 0.2707 | 0.9596 | 0.9597 | | 0.0091 | 17.6471 | 13800 | 0.2742 | 0.9597 | 0.9599 | | 0.0091 | 17.9028 | 14000 | 0.2777 | 0.9590 | 0.9591 | | 0.0057 | 18.1586 | 14200 | 0.2737 | 0.9596 | 0.9597 | | 0.0057 | 18.4143 | 14400 | 0.2731 | 0.9598 | 0.9599 | | 0.0057 | 18.6701 | 14600 | 0.2693 | 0.9606 | 0.9607 | | 0.0057 | 18.9258 | 14800 | 0.2754 | 0.9597 | 0.9598 | | 0.0074 | 19.1816 | 15000 | 0.2729 | 0.9602 | 0.9602 | | 0.0074 | 19.4373 | 15200 | 0.2784 | 0.9595 | 0.9596 | | 0.0074 | 19.6931 | 15400 | 0.2766 | 0.9598 | 0.9599 | | 0.0074 | 19.9488 | 15600 | 0.2741 | 0.9600 | 0.9602 | ### Framework versions - Transformers 4.41.2 - Pytorch 2.1.2 - Datasets 2.19.2 - Tokenizers 0.19.1