XMLRoberta_Lexical_Dataset59KCoDuoi
This model is a fine-tuned version of FacebookAI/xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.3668
- Accuracy: 0.9580
- F1: 0.9581
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
---|---|---|---|---|---|
No log | 0.2558 | 200 | 0.2520 | 0.9109 | 0.9116 |
No log | 0.5115 | 400 | 0.1839 | 0.9393 | 0.9398 |
No log | 0.7673 | 600 | 0.2109 | 0.9362 | 0.9369 |
0.2271 | 1.0230 | 800 | 0.1567 | 0.9510 | 0.9512 |
0.2271 | 1.2788 | 1000 | 0.1477 | 0.9500 | 0.9502 |
0.2271 | 1.5345 | 1200 | 0.1551 | 0.9526 | 0.9529 |
0.2271 | 1.7903 | 1400 | 0.1419 | 0.9538 | 0.9542 |
0.1372 | 2.0460 | 1600 | 0.1607 | 0.9550 | 0.9554 |
0.1372 | 2.3018 | 1800 | 0.1590 | 0.9568 | 0.9568 |
0.1372 | 2.5575 | 2000 | 0.1415 | 0.9595 | 0.9596 |
0.1372 | 2.8133 | 2200 | 0.1473 | 0.9580 | 0.9582 |
0.1109 | 3.0691 | 2400 | 0.1644 | 0.9538 | 0.9542 |
0.1109 | 3.3248 | 2600 | 0.1300 | 0.9605 | 0.9607 |
0.1109 | 3.5806 | 2800 | 0.1664 | 0.9588 | 0.9591 |
0.1109 | 3.8363 | 3000 | 0.1395 | 0.9570 | 0.9572 |
0.0958 | 4.0921 | 3200 | 0.1602 | 0.9602 | 0.9603 |
0.0958 | 4.3478 | 3400 | 0.1566 | 0.9615 | 0.9616 |
0.0958 | 4.6036 | 3600 | 0.1413 | 0.9583 | 0.9586 |
0.0958 | 4.8593 | 3800 | 0.1973 | 0.9582 | 0.9582 |
0.083 | 5.1151 | 4000 | 0.1469 | 0.9591 | 0.9594 |
0.083 | 5.3708 | 4200 | 0.1541 | 0.9603 | 0.9605 |
0.083 | 5.6266 | 4400 | 0.1676 | 0.9585 | 0.9587 |
0.083 | 5.8824 | 4600 | 0.1687 | 0.9602 | 0.9604 |
0.0734 | 6.1381 | 4800 | 0.1865 | 0.9591 | 0.9592 |
0.0734 | 6.3939 | 5000 | 0.1723 | 0.9569 | 0.9569 |
0.0734 | 6.6496 | 5200 | 0.1761 | 0.9587 | 0.9589 |
0.0734 | 6.9054 | 5400 | 0.1596 | 0.9613 | 0.9614 |
0.0607 | 7.1611 | 5600 | 0.2193 | 0.9586 | 0.9588 |
0.0607 | 7.4169 | 5800 | 0.1984 | 0.9595 | 0.9596 |
0.0607 | 7.6726 | 6000 | 0.1745 | 0.9587 | 0.9589 |
0.0607 | 7.9284 | 6200 | 0.1939 | 0.9614 | 0.9615 |
0.0547 | 8.1841 | 6400 | 0.2081 | 0.9591 | 0.9592 |
0.0547 | 8.4399 | 6600 | 0.2048 | 0.9599 | 0.9601 |
0.0547 | 8.6957 | 6800 | 0.2260 | 0.9563 | 0.9565 |
0.0547 | 8.9514 | 7000 | 0.1786 | 0.9598 | 0.9600 |
0.047 | 9.2072 | 7200 | 0.2181 | 0.9596 | 0.9597 |
0.047 | 9.4629 | 7400 | 0.2120 | 0.9602 | 0.9603 |
0.047 | 9.7187 | 7600 | 0.2266 | 0.9597 | 0.9597 |
0.047 | 9.9744 | 7800 | 0.2128 | 0.9581 | 0.9583 |
0.0409 | 10.2302 | 8000 | 0.2207 | 0.9607 | 0.9608 |
0.0409 | 10.4859 | 8200 | 0.2375 | 0.9597 | 0.9599 |
0.0409 | 10.7417 | 8400 | 0.2241 | 0.9592 | 0.9593 |
0.0368 | 10.9974 | 8600 | 0.2181 | 0.9613 | 0.9613 |
0.0368 | 11.2532 | 8800 | 0.2574 | 0.9598 | 0.9599 |
0.0368 | 11.5090 | 9000 | 0.2598 | 0.9602 | 0.9602 |
0.0368 | 11.7647 | 9200 | 0.2448 | 0.9592 | 0.9594 |
0.0309 | 12.0205 | 9400 | 0.2521 | 0.9593 | 0.9594 |
0.0309 | 12.2762 | 9600 | 0.2824 | 0.9599 | 0.9601 |
0.0309 | 12.5320 | 9800 | 0.2606 | 0.9600 | 0.9602 |
0.0309 | 12.7877 | 10000 | 0.2841 | 0.9610 | 0.9612 |
0.0256 | 13.0435 | 10200 | 0.2662 | 0.9590 | 0.9591 |
0.0256 | 13.2992 | 10400 | 0.2839 | 0.9582 | 0.9582 |
0.0256 | 13.5550 | 10600 | 0.3053 | 0.9579 | 0.9580 |
0.0256 | 13.8107 | 10800 | 0.2697 | 0.9573 | 0.9574 |
0.0229 | 14.0665 | 11000 | 0.2741 | 0.9583 | 0.9584 |
0.0229 | 14.3223 | 11200 | 0.2881 | 0.9596 | 0.9597 |
0.0229 | 14.5780 | 11400 | 0.2921 | 0.9586 | 0.9588 |
0.0229 | 14.8338 | 11600 | 0.3162 | 0.9598 | 0.9600 |
0.0196 | 15.0895 | 11800 | 0.2989 | 0.9575 | 0.9576 |
0.0196 | 15.3453 | 12000 | 0.3267 | 0.9568 | 0.9570 |
0.0196 | 15.6010 | 12200 | 0.3113 | 0.9593 | 0.9594 |
0.0196 | 15.8568 | 12400 | 0.3198 | 0.9595 | 0.9597 |
0.0167 | 16.1125 | 12600 | 0.3355 | 0.9580 | 0.9582 |
0.0167 | 16.3683 | 12800 | 0.3525 | 0.9566 | 0.9569 |
0.0167 | 16.6240 | 13000 | 0.3337 | 0.9582 | 0.9584 |
0.0167 | 16.8798 | 13200 | 0.3105 | 0.9583 | 0.9585 |
0.0139 | 17.1355 | 13400 | 0.3348 | 0.9597 | 0.9599 |
0.0139 | 17.3913 | 13600 | 0.3290 | 0.9592 | 0.9593 |
0.0139 | 17.6471 | 13800 | 0.3476 | 0.9587 | 0.9589 |
0.0139 | 17.9028 | 14000 | 0.3498 | 0.9583 | 0.9584 |
0.0131 | 18.1586 | 14200 | 0.3483 | 0.9590 | 0.9590 |
0.0131 | 18.4143 | 14400 | 0.3386 | 0.9587 | 0.9588 |
0.0131 | 18.6701 | 14600 | 0.3512 | 0.9581 | 0.9582 |
0.0131 | 18.9258 | 14800 | 0.3627 | 0.9581 | 0.9582 |
0.01 | 19.1816 | 15000 | 0.3664 | 0.9572 | 0.9574 |
0.01 | 19.4373 | 15200 | 0.3688 | 0.9576 | 0.9578 |
0.01 | 19.6931 | 15400 | 0.3672 | 0.9579 | 0.9580 |
0.01 | 19.9488 | 15600 | 0.3668 | 0.9580 | 0.9581 |
Framework versions
- Transformers 4.41.2
- Pytorch 2.1.2
- Datasets 2.19.2
- Tokenizers 0.19.1
- Downloads last month
- 1