|
--- |
|
license: apache-2.0 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- crows_pairs |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: bert-large-uncased_crows_pairs_finetuned |
|
results: |
|
- task: |
|
name: Text Classification |
|
type: text-classification |
|
dataset: |
|
name: crows_pairs |
|
type: crows_pairs |
|
config: crows_pairs |
|
split: test |
|
args: crows_pairs |
|
metrics: |
|
- name: Accuracy |
|
type: accuracy |
|
value: 0.7483443708609272 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# bert-large-uncased_crows_pairs_finetuned |
|
|
|
This model is a fine-tuned version of [bert-large-uncased](https://huggingface.co/bert-large-uncased) on the crows_pairs dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.8687 |
|
- Accuracy: 0.7483 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 64 |
|
- eval_batch_size: 64 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 30 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:--------:| |
|
| 0.7241 | 0.53 | 10 | 0.7015 | 0.4868 | |
|
| 0.6975 | 1.05 | 20 | 0.7124 | 0.4934 | |
|
| 0.6992 | 1.58 | 30 | 0.7102 | 0.4901 | |
|
| 0.6941 | 2.11 | 40 | 0.6890 | 0.5430 | |
|
| 0.665 | 2.63 | 50 | 0.6981 | 0.5464 | |
|
| 0.5656 | 3.16 | 60 | 0.5434 | 0.7119 | |
|
| 0.3695 | 3.68 | 70 | 0.7431 | 0.7152 | |
|
| 0.3913 | 4.21 | 80 | 0.7834 | 0.7185 | |
|
| 0.2617 | 4.74 | 90 | 0.6394 | 0.7318 | |
|
| 0.1225 | 5.26 | 100 | 0.8882 | 0.6921 | |
|
| 0.1549 | 5.79 | 110 | 0.8629 | 0.7119 | |
|
| 0.1094 | 6.32 | 120 | 1.0113 | 0.7185 | |
|
| 0.0418 | 6.84 | 130 | 1.2568 | 0.7219 | |
|
| 0.023 | 7.37 | 140 | 1.3223 | 0.7417 | |
|
| 0.045 | 7.89 | 150 | 1.5015 | 0.7086 | |
|
| 0.0216 | 8.42 | 160 | 1.1833 | 0.7550 | |
|
| 0.0084 | 8.95 | 170 | 1.4125 | 0.7318 | |
|
| 0.0198 | 9.47 | 180 | 1.5301 | 0.7152 | |
|
| 0.0061 | 10.0 | 190 | 1.3163 | 0.7483 | |
|
| 0.0041 | 10.53 | 200 | 1.3083 | 0.7517 | |
|
| 0.0046 | 11.05 | 210 | 1.4028 | 0.7616 | |
|
| 0.0034 | 11.58 | 220 | 1.5256 | 0.7583 | |
|
| 0.0012 | 12.11 | 230 | 1.6067 | 0.7583 | |
|
| 0.001 | 12.63 | 240 | 1.6199 | 0.7649 | |
|
| 0.005 | 13.16 | 250 | 1.7140 | 0.7384 | |
|
| 0.0031 | 13.68 | 260 | 1.7680 | 0.7318 | |
|
| 0.0008 | 14.21 | 270 | 1.7353 | 0.7252 | |
|
| 0.0026 | 14.74 | 280 | 1.7242 | 0.7450 | |
|
| 0.0013 | 15.26 | 290 | 1.7290 | 0.7483 | |
|
| 0.0001 | 15.79 | 300 | 1.7421 | 0.7450 | |
|
| 0.0008 | 16.32 | 310 | 1.7536 | 0.7450 | |
|
| 0.0013 | 16.84 | 320 | 1.7588 | 0.7483 | |
|
| 0.0014 | 17.37 | 330 | 1.8153 | 0.7417 | |
|
| 0.0025 | 17.89 | 340 | 1.8432 | 0.7450 | |
|
| 0.0024 | 18.42 | 350 | 1.8597 | 0.7351 | |
|
| 0.0022 | 18.95 | 360 | 1.8676 | 0.7384 | |
|
| 0.0001 | 19.47 | 370 | 1.8602 | 0.7417 | |
|
| 0.0032 | 20.0 | 380 | 1.8600 | 0.7450 | |
|
| 0.0017 | 20.53 | 390 | 1.8576 | 0.7417 | |
|
| 0.0022 | 21.05 | 400 | 1.8603 | 0.7417 | |
|
| 0.0024 | 21.58 | 410 | 1.8649 | 0.7417 | |
|
| 0.002 | 22.11 | 420 | 1.8704 | 0.7417 | |
|
| 0.0008 | 22.63 | 430 | 1.8764 | 0.7417 | |
|
| 0.0019 | 23.16 | 440 | 1.8914 | 0.7417 | |
|
| 0.0015 | 23.68 | 450 | 1.9026 | 0.7384 | |
|
| 0.0014 | 24.21 | 460 | 1.9146 | 0.7384 | |
|
| 0.0017 | 24.74 | 470 | 1.9258 | 0.7384 | |
|
| 0.0027 | 25.26 | 480 | 1.9280 | 0.7384 | |
|
| 0.0027 | 25.79 | 490 | 1.9285 | 0.7384 | |
|
| 0.001 | 26.32 | 500 | 1.9236 | 0.7384 | |
|
| 0.0034 | 26.84 | 510 | 1.8905 | 0.7450 | |
|
| 0.0013 | 27.37 | 520 | 1.8730 | 0.7417 | |
|
| 0.0016 | 27.89 | 530 | 1.8687 | 0.7450 | |
|
| 0.0007 | 28.42 | 540 | 1.8681 | 0.7483 | |
|
| 0.0042 | 28.95 | 550 | 1.8683 | 0.7483 | |
|
| 0.0009 | 29.47 | 560 | 1.8686 | 0.7483 | |
|
| 0.0018 | 30.0 | 570 | 1.8687 | 0.7483 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.26.1 |
|
- Pytorch 1.13.1 |
|
- Datasets 2.10.1 |
|
- Tokenizers 0.13.2 |
|
|
