ms-coa-legal-bert-base-uncased
This model is a fine-tuned version of nlpaueb/legal-bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.1135
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.5686 | 0.06 | 323 | 1.3832 |
| 1.461 | 0.12 | 646 | 1.3481 |
| 1.4377 | 0.19 | 969 | 1.3334 |
| 1.4151 | 0.25 | 1292 | 1.3130 |
| 1.3889 | 0.31 | 1615 | 1.2913 |
| 1.385 | 0.37 | 1938 | 1.2890 |
| 1.369 | 0.44 | 2261 | 1.2747 |
| 1.3597 | 0.5 | 2584 | 1.2654 |
| 1.367 | 0.56 | 2907 | 1.2632 |
| 1.3519 | 0.62 | 3230 | 1.2561 |
| 1.3294 | 0.69 | 3553 | 1.2455 |
| 1.326 | 0.75 | 3876 | 1.2428 |
| 1.3275 | 0.81 | 4199 | 1.2332 |
| 1.3174 | 0.87 | 4522 | 1.2276 |
| 1.3079 | 0.94 | 4845 | 1.2261 |
| 1.2978 | 1.0 | 5168 | 1.2142 |
| 1.2863 | 1.06 | 5491 | 1.2089 |
| 1.2832 | 1.12 | 5814 | 1.2022 |
| 1.2774 | 1.19 | 6137 | 1.2020 |
| 1.2851 | 1.25 | 6460 | 1.1975 |
| 1.2628 | 1.31 | 6783 | 1.1924 |
| 1.2756 | 1.37 | 7106 | 1.1918 |
| 1.2643 | 1.44 | 7429 | 1.1905 |
| 1.2578 | 1.5 | 7752 | 1.1850 |
| 1.2606 | 1.56 | 8075 | 1.1812 |
| 1.251 | 1.62 | 8398 | 1.1779 |
| 1.2477 | 1.69 | 8721 | 1.1731 |
| 1.2518 | 1.75 | 9044 | 1.1721 |
| 1.2428 | 1.81 | 9367 | 1.1723 |
| 1.2294 | 1.87 | 9690 | 1.1684 |
| 1.2357 | 1.94 | 10013 | 1.1630 |
| 1.2456 | 2.0 | 10336 | 1.1610 |
| 1.2196 | 2.06 | 10659 | 1.1588 |
| 1.2234 | 2.12 | 10982 | 1.1557 |
| 1.2353 | 2.19 | 11305 | 1.1530 |
| 1.2205 | 2.25 | 11628 | 1.1483 |
| 1.2243 | 2.31 | 11951 | 1.1539 |
| 1.226 | 2.37 | 12274 | 1.1519 |
| 1.2193 | 2.44 | 12597 | 1.1514 |
| 1.2152 | 2.5 | 12920 | 1.1472 |
| 1.2137 | 2.56 | 13243 | 1.1410 |
| 1.211 | 2.62 | 13566 | 1.1389 |
| 1.2113 | 2.69 | 13889 | 1.1443 |
| 1.206 | 2.75 | 14212 | 1.1448 |
| 1.213 | 2.81 | 14535 | 1.1375 |
| 1.1941 | 2.87 | 14858 | 1.1323 |
| 1.2089 | 2.94 | 15181 | 1.1364 |
| 1.2 | 3.0 | 15504 | 1.1320 |
| 1.1935 | 3.06 | 15827 | 1.1321 |
| 1.1874 | 3.12 | 16150 | 1.1296 |
| 1.1934 | 3.19 | 16473 | 1.1315 |
| 1.1928 | 3.25 | 16796 | 1.1271 |
| 1.1853 | 3.31 | 17119 | 1.1304 |
| 1.1931 | 3.37 | 17442 | 1.1266 |
| 1.1841 | 3.44 | 17765 | 1.1232 |
| 1.1734 | 3.5 | 18088 | 1.1300 |
| 1.18 | 3.56 | 18411 | 1.1254 |
| 1.174 | 3.62 | 18734 | 1.1236 |
| 1.1822 | 3.69 | 19057 | 1.1206 |
| 1.177 | 3.75 | 19380 | 1.1187 |
| 1.1756 | 3.81 | 19703 | 1.1183 |
| 1.1748 | 3.87 | 20026 | 1.1134 |
| 1.1693 | 3.94 | 20349 | 1.1138 |
| 1.1768 | 4.0 | 20672 | 1.1135 |
Framework versions
- Transformers 4.34.1
- Pytorch 1.12.1+cu113
- Datasets 2.8.0
- Tokenizers 0.14.1
- Downloads last month
- 1