legalectra-small-spanish-becasv3-6
This model is a fine-tuned version of mrm8488/legalectra-small-spanish on the becasv2 dataset. It achieves the following results on the evaluation set:
- Loss: 3.8441
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 150
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 1.0 | 5 | 5.6469 |
| No log | 2.0 | 10 | 5.5104 |
| No log | 3.0 | 15 | 5.4071 |
| No log | 4.0 | 20 | 5.3313 |
| No log | 5.0 | 25 | 5.2629 |
| No log | 6.0 | 30 | 5.1972 |
| No log | 7.0 | 35 | 5.1336 |
| No log | 8.0 | 40 | 5.0667 |
| No log | 9.0 | 45 | 5.0030 |
| No log | 10.0 | 50 | 4.9302 |
| No log | 11.0 | 55 | 4.8646 |
| No log | 12.0 | 60 | 4.7963 |
| No log | 13.0 | 65 | 4.7328 |
| No log | 14.0 | 70 | 4.6735 |
| No log | 15.0 | 75 | 4.6258 |
| No log | 16.0 | 80 | 4.5869 |
| No log | 17.0 | 85 | 4.5528 |
| No log | 18.0 | 90 | 4.5177 |
| No log | 19.0 | 95 | 4.4916 |
| No log | 20.0 | 100 | 4.4685 |
| No log | 21.0 | 105 | 4.4371 |
| No log | 22.0 | 110 | 4.4271 |
| No log | 23.0 | 115 | 4.3905 |
| No log | 24.0 | 120 | 4.3931 |
| No log | 25.0 | 125 | 4.3902 |
| No log | 26.0 | 130 | 4.3772 |
| No log | 27.0 | 135 | 4.3981 |
| No log | 28.0 | 140 | 4.4463 |
| No log | 29.0 | 145 | 4.4501 |
| No log | 30.0 | 150 | 4.4654 |
| No log | 31.0 | 155 | 4.4069 |
| No log | 32.0 | 160 | 4.4108 |
| No log | 33.0 | 165 | 4.4394 |
| No log | 34.0 | 170 | 4.4320 |
| No log | 35.0 | 175 | 4.3541 |
| No log | 36.0 | 180 | 4.4534 |
| No log | 37.0 | 185 | 4.2616 |
| No log | 38.0 | 190 | 4.2474 |
| No log | 39.0 | 195 | 4.4358 |
| No log | 40.0 | 200 | 4.3060 |
| No log | 41.0 | 205 | 4.1866 |
| No log | 42.0 | 210 | 4.2735 |
| No log | 43.0 | 215 | 4.2739 |
| No log | 44.0 | 220 | 4.1812 |
| No log | 45.0 | 225 | 4.2484 |
| No log | 46.0 | 230 | 4.3706 |
| No log | 47.0 | 235 | 4.3487 |
| No log | 48.0 | 240 | 4.2805 |
| No log | 49.0 | 245 | 4.3180 |
| No log | 50.0 | 250 | 4.3574 |
| No log | 51.0 | 255 | 4.2823 |
| No log | 52.0 | 260 | 4.0643 |
| No log | 53.0 | 265 | 4.0729 |
| No log | 54.0 | 270 | 4.2368 |
| No log | 55.0 | 275 | 4.2845 |
| No log | 56.0 | 280 | 4.1009 |
| No log | 57.0 | 285 | 4.0629 |
| No log | 58.0 | 290 | 4.1250 |
| No log | 59.0 | 295 | 4.2048 |
| No log | 60.0 | 300 | 4.2412 |
| No log | 61.0 | 305 | 4.1653 |
| No log | 62.0 | 310 | 4.1433 |
| No log | 63.0 | 315 | 4.1309 |
| No log | 64.0 | 320 | 4.1381 |
| No log | 65.0 | 325 | 4.2162 |
| No log | 66.0 | 330 | 4.1858 |
| No log | 67.0 | 335 | 4.1342 |
| No log | 68.0 | 340 | 4.1247 |
| No log | 69.0 | 345 | 4.1701 |
| No log | 70.0 | 350 | 4.1915 |
| No log | 71.0 | 355 | 4.1356 |
| No log | 72.0 | 360 | 4.1766 |
| No log | 73.0 | 365 | 4.1296 |
| No log | 74.0 | 370 | 4.0594 |
| No log | 75.0 | 375 | 4.0601 |
| No log | 76.0 | 380 | 4.0328 |
| No log | 77.0 | 385 | 3.9978 |
| No log | 78.0 | 390 | 4.0070 |
| No log | 79.0 | 395 | 4.0519 |
| No log | 80.0 | 400 | 4.1000 |
| No log | 81.0 | 405 | 3.9550 |
| No log | 82.0 | 410 | 3.9159 |
| No log | 83.0 | 415 | 3.9494 |
| No log | 84.0 | 420 | 4.0546 |
| No log | 85.0 | 425 | 4.2223 |
| No log | 86.0 | 430 | 4.2665 |
| No log | 87.0 | 435 | 3.8892 |
| No log | 88.0 | 440 | 3.7763 |
| No log | 89.0 | 445 | 3.8576 |
| No log | 90.0 | 450 | 4.0089 |
| No log | 91.0 | 455 | 4.1495 |
| No log | 92.0 | 460 | 4.1545 |
| No log | 93.0 | 465 | 4.0164 |
| No log | 94.0 | 470 | 3.9175 |
| No log | 95.0 | 475 | 3.9308 |
| No log | 96.0 | 480 | 3.9658 |
| No log | 97.0 | 485 | 3.9856 |
| No log | 98.0 | 490 | 3.9691 |
| No log | 99.0 | 495 | 3.9082 |
| 3.2873 | 100.0 | 500 | 3.8736 |
| 3.2873 | 101.0 | 505 | 3.8963 |
| 3.2873 | 102.0 | 510 | 3.9391 |
| 3.2873 | 103.0 | 515 | 3.9408 |
| 3.2873 | 104.0 | 520 | 3.9075 |
| 3.2873 | 105.0 | 525 | 3.8258 |
| 3.2873 | 106.0 | 530 | 3.7917 |
| 3.2873 | 107.0 | 535 | 3.7981 |
| 3.2873 | 108.0 | 540 | 3.8272 |
| 3.2873 | 109.0 | 545 | 3.8655 |
| 3.2873 | 110.0 | 550 | 3.8234 |
| 3.2873 | 111.0 | 555 | 3.7126 |
| 3.2873 | 112.0 | 560 | 3.6981 |
| 3.2873 | 113.0 | 565 | 3.7327 |
| 3.2873 | 114.0 | 570 | 3.8470 |
| 3.2873 | 115.0 | 575 | 4.0036 |
| 3.2873 | 116.0 | 580 | 4.0412 |
| 3.2873 | 117.0 | 585 | 4.0487 |
| 3.2873 | 118.0 | 590 | 4.0524 |
| 3.2873 | 119.0 | 595 | 4.0375 |
| 3.2873 | 120.0 | 600 | 3.9971 |
| 3.2873 | 121.0 | 605 | 3.8959 |
| 3.2873 | 122.0 | 610 | 3.8834 |
| 3.2873 | 123.0 | 615 | 3.9279 |
| 3.2873 | 124.0 | 620 | 3.9374 |
| 3.2873 | 125.0 | 625 | 3.9515 |
| 3.2873 | 126.0 | 630 | 3.9625 |
| 3.2873 | 127.0 | 635 | 3.9635 |
| 3.2873 | 128.0 | 640 | 3.9596 |
| 3.2873 | 129.0 | 645 | 3.8871 |
| 3.2873 | 130.0 | 650 | 3.8307 |
| 3.2873 | 131.0 | 655 | 3.8318 |
| 3.2873 | 132.0 | 660 | 3.8403 |
| 3.2873 | 133.0 | 665 | 3.8560 |
| 3.2873 | 134.0 | 670 | 3.8650 |
| 3.2873 | 135.0 | 675 | 3.8734 |
| 3.2873 | 136.0 | 680 | 3.8756 |
| 3.2873 | 137.0 | 685 | 3.8613 |
| 3.2873 | 138.0 | 690 | 3.8447 |
| 3.2873 | 139.0 | 695 | 3.8362 |
| 3.2873 | 140.0 | 700 | 3.8328 |
| 3.2873 | 141.0 | 705 | 3.8350 |
| 3.2873 | 142.0 | 710 | 3.8377 |
| 3.2873 | 143.0 | 715 | 3.8399 |
| 3.2873 | 144.0 | 720 | 3.8414 |
| 3.2873 | 145.0 | 725 | 3.8422 |
| 3.2873 | 146.0 | 730 | 3.8435 |
| 3.2873 | 147.0 | 735 | 3.8437 |
| 3.2873 | 148.0 | 740 | 3.8437 |
| 3.2873 | 149.0 | 745 | 3.8440 |
| 3.2873 | 150.0 | 750 | 3.8441 |
Framework versions
- Transformers 4.20.1
- Pytorch 1.11.0+cu113
- Datasets 2.3.2
- Tokenizers 0.12.1
- Downloads last month
- 8