---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- finer-139
metrics:
- precision
- recall
- f1
- accuracy
base_model: google/bert_uncased_L-2_H-128_A-2
model-index:
- name: bertiny-finetuned-finer-full
  results:
  - task:
      type: token-classification
      name: Token Classification
    dataset:
      name: finer-139
      type: finer-139
      args: finer-139
    metrics:
    - type: precision
      value: 0.555368475586064
      name: Precision
    - type: recall
      value: 0.5164398410213176
      name: Recall
    - type: f1
      value: 0.5351972041937094
      name: F1
    - type: accuracy
      value: 0.988733187308122
      name: Accuracy
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bertiny-finetuned-finer-full

This model is a fine-tuned version of [google/bert_uncased_L-2_H-128_A-2](https://huggingface.co/google/bert_uncased_L-2_H-128_A-2) on the 10% of finer-139 dataset for 40 epochs according to paper.
It achieves the following results on the evaluation set:
- Loss: 0.0788
- Precision: 0.5554
- Recall: 0.5164
- F1: 0.5352
- Accuracy: 0.9887

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 40

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:------:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.0852        | 1.0   | 11255  | 0.0929          | 1.0       | 0.0001 | 0.0002 | 0.9843   |
| 0.08          | 2.0   | 22510  | 0.0840          | 0.4626    | 0.0730 | 0.1261 | 0.9851   |
| 0.0759        | 3.0   | 33765  | 0.0750          | 0.5113    | 0.2035 | 0.2912 | 0.9865   |
| 0.0569        | 4.0   | 45020  | 0.0673          | 0.4973    | 0.3281 | 0.3953 | 0.9872   |
| 0.0488        | 5.0   | 56275  | 0.0635          | 0.5289    | 0.3749 | 0.4388 | 0.9878   |
| 0.0422        | 6.0   | 67530  | 0.0606          | 0.5258    | 0.4068 | 0.4587 | 0.9880   |
| 0.0364        | 7.0   | 78785  | 0.0600          | 0.5588    | 0.4186 | 0.4787 | 0.9883   |
| 0.0307        | 8.0   | 90040  | 0.0589          | 0.5223    | 0.4916 | 0.5065 | 0.9883   |
| 0.0284        | 9.0   | 101295 | 0.0595          | 0.5588    | 0.4813 | 0.5171 | 0.9887   |
| 0.0255        | 10.0  | 112550 | 0.0597          | 0.5606    | 0.4944 | 0.5254 | 0.9888   |
| 0.0223        | 11.0  | 123805 | 0.0600          | 0.5533    | 0.4998 | 0.5252 | 0.9888   |
| 0.0228        | 12.0  | 135060 | 0.0608          | 0.5290    | 0.5228 | 0.5259 | 0.9885   |
| 0.0225        | 13.0  | 146315 | 0.0612          | 0.5480    | 0.5111 | 0.5289 | 0.9887   |
| 0.0204        | 14.0  | 157570 | 0.0634          | 0.5646    | 0.5120 | 0.5370 | 0.9890   |
| 0.0176        | 15.0  | 168825 | 0.0639          | 0.5611    | 0.5135 | 0.5363 | 0.9889   |
| 0.0167        | 16.0  | 180080 | 0.0647          | 0.5631    | 0.5120 | 0.5363 | 0.9888   |
| 0.0161        | 17.0  | 191335 | 0.0665          | 0.5607    | 0.5081 | 0.5331 | 0.9889   |
| 0.0145        | 18.0  | 202590 | 0.0673          | 0.5437    | 0.5280 | 0.5357 | 0.9887   |
| 0.0166        | 19.0  | 213845 | 0.0687          | 0.5722    | 0.5008 | 0.5341 | 0.9889   |
| 0.0155        | 20.0  | 225100 | 0.0685          | 0.5325    | 0.5337 | 0.5331 | 0.9885   |
| 0.0142        | 21.0  | 236355 | 0.0705          | 0.5626    | 0.5166 | 0.5386 | 0.9890   |
| 0.0127        | 22.0  | 247610 | 0.0694          | 0.5426    | 0.5358 | 0.5392 | 0.9887   |
| 0.0112        | 23.0  | 258865 | 0.0721          | 0.5591    | 0.5129 | 0.5351 | 0.9888   |
| 0.0123        | 24.0  | 270120 | 0.0733          | 0.5715    | 0.5081 | 0.5380 | 0.9889   |
| 0.0116        | 25.0  | 281375 | 0.0735          | 0.5621    | 0.5123 | 0.5361 | 0.9888   |
| 0.0112        | 26.0  | 292630 | 0.0739          | 0.5634    | 0.5181 | 0.5398 | 0.9889   |
| 0.0108        | 27.0  | 303885 | 0.0753          | 0.5548    | 0.5155 | 0.5344 | 0.9887   |
| 0.0125        | 28.0  | 315140 | 0.0746          | 0.5507    | 0.5221 | 0.5360 | 0.9886   |
| 0.0093        | 29.0  | 326395 | 0.0762          | 0.5602    | 0.5156 | 0.5370 | 0.9888   |
| 0.0094        | 30.0  | 337650 | 0.0762          | 0.5625    | 0.5157 | 0.5381 | 0.9889   |
| 0.0117        | 31.0  | 348905 | 0.0767          | 0.5519    | 0.5195 | 0.5352 | 0.9887   |
| 0.0091        | 32.0  | 360160 | 0.0772          | 0.5501    | 0.5198 | 0.5345 | 0.9887   |
| 0.0109        | 33.0  | 371415 | 0.0775          | 0.5635    | 0.5097 | 0.5353 | 0.9888   |
| 0.0094        | 34.0  | 382670 | 0.0776          | 0.5467    | 0.5216 | 0.5339 | 0.9887   |
| 0.009         | 35.0  | 393925 | 0.0782          | 0.5601    | 0.5139 | 0.5360 | 0.9889   |
| 0.0093        | 36.0  | 405180 | 0.0780          | 0.5568    | 0.5156 | 0.5354 | 0.9888   |
| 0.0087        | 37.0  | 416435 | 0.0783          | 0.5588    | 0.5143 | 0.5356 | 0.9888   |
| 0.009         | 38.0  | 427690 | 0.0785          | 0.5483    | 0.5178 | 0.5326 | 0.9887   |
| 0.0094        | 39.0  | 438945 | 0.0787          | 0.5541    | 0.5154 | 0.5340 | 0.9887   |
| 0.0088        | 40.0  | 450200 | 0.0788          | 0.5554    | 0.5164 | 0.5352 | 0.9887   |


### Framework versions

- Transformers 4.20.1
- Pytorch 1.12.0+cu113
- Datasets 2.3.2
- Tokenizers 0.12.1