Hugging Face's logo

IntelLabs
/
eftnas-s1-bert-base

English
Model card Files
eftnas-s1-bert-base / eftnas-s1-bert-base-rte /README.md
jinjieyuan's picture
jinjieyuan
Upload model
1960aa4
preview code
|
raw
history blame
12.8 kB
metadata
language:
  - en
license: apache-2.0
tags:
  - generated_from_trainer
datasets:
  - glue
metrics:
  - accuracy
model-index:
  - name: tryv3_16epochs
    results:
      - task:
          name: Text Classification
          type: text-classification
        dataset:
          name: GLUE RTE
          type: glue
          config: rte
          split: validation
          args: rte
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.6498194945848376

tryv3_16epochs

This model is a fine-tuned version of bert-base-uncased on the GLUE RTE dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0256
  • Accuracy: 0.6498

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 32
  • eval_batch_size: 128
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 1.0 78 0.6907 0.5126 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
No log 1.0 78 0.6698 0.6318 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
No log 2.0 156 0.7006 0.5884 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
No log 2.0 156 0.6683 0.6715 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
No log 3.0 234 0.7282 0.5957 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
No log 3.0 234 0.7189 0.7004 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
No log 4.0 312 0.7194 0.6534 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
No log 4.0 312 0.7805 0.7004 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
No log 5.0 390 0.8791 0.6354 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
No log 5.0 390 0.8328 0.7076 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
No log 6.0 468 1.0037 0.6173 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
No log 6.0 468 0.8701 0.6895 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.5371 7.0 546 0.9121 0.6245 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.5371 7.0 546 0.8220 0.6859 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.5371 8.0 624 1.0092 0.6245 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.5371 8.0 624 0.8341 0.6823 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.5371 9.0 702 0.9687 0.6426 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.5371 9.0 702 0.8538 0.6643 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.5371 10.0 780 1.0111 0.6354 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.5371 10.0 780 0.8117 0.6968 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.5371 11.0 858 0.9616 0.6498 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.5371 11.0 858 0.8113 0.6895 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.5371 12.0 936 0.9934 0.6462 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.5371 12.0 936 0.8179 0.6895 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.1174 13.0 1014 1.0097 0.6318 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.1174 13.0 1014 0.8191 0.7004 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.1174 14.0 1092 1.0019 0.6462 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.1174 14.0 1092 0.8157 0.7004 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.1174 15.0 1170 1.0127 0.6318 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.1174 15.0 1170 0.8178 0.6895 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])
0.1174 16.0 1248 1.0095 0.6462 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 576, 1: 448, 2: 576, 3: 768, 4: 704, 5: 704, 6: 768, 7: 576, 8: 704, 9: 704, 10: 512, 11: 640, 12: 608, 13: 571, 14: 589, 15: 542, 16: 576, 17: 589, 18: 568, 19: 537, 20: 562, 21: 453, 22: 376, 23: 147})])
0.1174 16.0 1248 0.8178 0.6895 OrderedDict([(<ElasticityDim.WIDTH: 'width'>, {0: 768, 1: 768, 2: 768, 3: 768, 4: 768, 5: 768, 6: 768, 7: 768, 8: 768, 9: 768, 10: 768, 11: 768, 12: 3072, 13: 3072, 14: 3072, 15: 3072, 16: 3072, 17: 3072, 18: 3072, 19: 3072, 20: 3072, 21: 3072, 22: 3072, 23: 3072})])

Framework versions

  • Transformers 4.29.1
  • Pytorch 1.12.1
  • Datasets 2.13.1
  • Tokenizers 0.13.3