---
license: apache-2.0
base_model: google-bert/bert-base-uncased
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: 20240327180321_slow_hinton
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 20240327180321_slow_hinton

This model is a fine-tuned version of [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0488
- Precision: 0.9507
- Recall: 0.9581
- F1: 0.9544
- Accuracy: 0.9830

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 32
- eval_batch_size: 32
- seed: 69
- gradient_accumulation_steps: 8
- total_train_batch_size: 256
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 350
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 0.095         | 0.09  | 300   | 0.0845          | 0.9071    | 0.9202 | 0.9136 | 0.9668   |
| 0.0884        | 0.18  | 600   | 0.0782          | 0.9112    | 0.9274 | 0.9192 | 0.9689   |
| 0.0861        | 0.26  | 900   | 0.0761          | 0.9139    | 0.9294 | 0.9215 | 0.9698   |
| 0.082         | 0.35  | 1200  | 0.0742          | 0.9171    | 0.9322 | 0.9246 | 0.9711   |
| 0.0794        | 0.44  | 1500  | 0.0708          | 0.9229    | 0.9330 | 0.9279 | 0.9725   |
| 0.0788        | 0.53  | 1800  | 0.0699          | 0.9239    | 0.9339 | 0.9289 | 0.9729   |
| 0.078         | 0.62  | 2100  | 0.0701          | 0.9224    | 0.9339 | 0.9281 | 0.9726   |
| 0.0785        | 0.71  | 2400  | 0.0698          | 0.9278    | 0.9286 | 0.9282 | 0.9727   |
| 0.0768        | 0.79  | 2700  | 0.0686          | 0.9285    | 0.9326 | 0.9306 | 0.9736   |
| 0.0764        | 0.88  | 3000  | 0.0694          | 0.9166    | 0.9418 | 0.9290 | 0.9727   |
| 0.0754        | 0.97  | 3300  | 0.0674          | 0.9289    | 0.9341 | 0.9315 | 0.9740   |
| 0.0687        | 1.06  | 3600  | 0.0665          | 0.9304    | 0.9359 | 0.9332 | 0.9746   |
| 0.0697        | 1.15  | 3900  | 0.0664          | 0.9256    | 0.9410 | 0.9332 | 0.9744   |
| 0.0682        | 1.24  | 4200  | 0.0651          | 0.9258    | 0.9418 | 0.9337 | 0.9746   |
| 0.0679        | 1.32  | 4500  | 0.0637          | 0.9296    | 0.9425 | 0.9360 | 0.9757   |
| 0.0685        | 1.41  | 4800  | 0.0640          | 0.9288    | 0.9428 | 0.9357 | 0.9755   |
| 0.0662        | 1.5   | 5100  | 0.0627          | 0.9336    | 0.9394 | 0.9365 | 0.9760   |
| 0.0655        | 1.59  | 5400  | 0.0617          | 0.9334    | 0.9422 | 0.9378 | 0.9764   |
| 0.0656        | 1.68  | 5700  | 0.0621          | 0.9298    | 0.9458 | 0.9377 | 0.9763   |
| 0.065         | 1.77  | 6000  | 0.0610          | 0.9352    | 0.9419 | 0.9386 | 0.9768   |
| 0.0647        | 1.85  | 6300  | 0.0597          | 0.9341    | 0.9465 | 0.9403 | 0.9774   |
| 0.0629        | 1.94  | 6600  | 0.0591          | 0.9342    | 0.9457 | 0.9399 | 0.9772   |
| 0.0557        | 2.03  | 6900  | 0.0592          | 0.9375    | 0.9455 | 0.9415 | 0.9779   |
| 0.0563        | 2.12  | 7200  | 0.0598          | 0.9355    | 0.9454 | 0.9404 | 0.9774   |
| 0.0564        | 2.21  | 7500  | 0.0573          | 0.9375    | 0.9483 | 0.9428 | 0.9783   |
| 0.0574        | 2.3   | 7800  | 0.0571          | 0.9368    | 0.9490 | 0.9429 | 0.9783   |
| 0.0564        | 2.38  | 8100  | 0.0578          | 0.9375    | 0.9482 | 0.9428 | 0.9783   |
| 0.0553        | 2.47  | 8400  | 0.0574          | 0.9387    | 0.9472 | 0.9429 | 0.9785   |
| 0.0557        | 2.56  | 8700  | 0.0564          | 0.9378    | 0.9505 | 0.9441 | 0.9788   |
| 0.0554        | 2.65  | 9000  | 0.0557          | 0.9410    | 0.9472 | 0.9441 | 0.9789   |
| 0.0542        | 2.74  | 9300  | 0.0545          | 0.9409    | 0.9516 | 0.9462 | 0.9796   |
| 0.0533        | 2.83  | 9600  | 0.0540          | 0.9430    | 0.9501 | 0.9465 | 0.9799   |
| 0.0523        | 2.91  | 9900  | 0.0538          | 0.9388    | 0.9523 | 0.9455 | 0.9794   |
| 0.0509        | 3.0   | 10200 | 0.0547          | 0.9430    | 0.9503 | 0.9466 | 0.9798   |
| 0.0459        | 3.09  | 10500 | 0.0538          | 0.9428    | 0.9512 | 0.9470 | 0.9801   |
| 0.0443        | 3.18  | 10800 | 0.0549          | 0.9438    | 0.9496 | 0.9467 | 0.9800   |
| 0.0458        | 3.27  | 11100 | 0.0536          | 0.9440    | 0.9516 | 0.9478 | 0.9804   |
| 0.0445        | 3.36  | 11400 | 0.0523          | 0.9451    | 0.9509 | 0.9480 | 0.9805   |
| 0.0449        | 3.44  | 11700 | 0.0513          | 0.9453    | 0.9527 | 0.9490 | 0.9808   |
| 0.0442        | 3.53  | 12000 | 0.0518          | 0.9477    | 0.9513 | 0.9495 | 0.9811   |
| 0.0441        | 3.62  | 12300 | 0.0511          | 0.9447    | 0.9551 | 0.9499 | 0.9811   |
| 0.0439        | 3.71  | 12600 | 0.0503          | 0.9465    | 0.9556 | 0.9510 | 0.9815   |
| 0.0442        | 3.8   | 12900 | 0.0502          | 0.9466    | 0.9538 | 0.9502 | 0.9813   |
| 0.0431        | 3.88  | 13200 | 0.0503          | 0.9473    | 0.9549 | 0.9511 | 0.9817   |
| 0.0429        | 3.97  | 13500 | 0.0491          | 0.9473    | 0.9559 | 0.9516 | 0.9819   |
| 0.0356        | 4.06  | 13800 | 0.0522          | 0.9465    | 0.9566 | 0.9515 | 0.9818   |
| 0.0354        | 4.15  | 14100 | 0.0518          | 0.9489    | 0.9560 | 0.9524 | 0.9822   |
| 0.0357        | 4.24  | 14400 | 0.0509          | 0.9485    | 0.9565 | 0.9525 | 0.9822   |
| 0.0353        | 4.33  | 14700 | 0.0507          | 0.9492    | 0.9563 | 0.9527 | 0.9823   |
| 0.0352        | 4.41  | 15000 | 0.0498          | 0.9497    | 0.9572 | 0.9534 | 0.9826   |
| 0.0352        | 4.5   | 15300 | 0.0492          | 0.9496    | 0.9577 | 0.9536 | 0.9826   |
| 0.0341        | 4.59  | 15600 | 0.0493          | 0.9494    | 0.9583 | 0.9538 | 0.9827   |
| 0.034         | 4.68  | 15900 | 0.0495          | 0.9504    | 0.9576 | 0.9540 | 0.9828   |
| 0.0334        | 4.77  | 16200 | 0.0493          | 0.9501    | 0.9584 | 0.9542 | 0.9829   |
| 0.0335        | 4.86  | 16500 | 0.0493          | 0.9509    | 0.9574 | 0.9541 | 0.9828   |
| 0.0338        | 4.94  | 16800 | 0.0488          | 0.9507    | 0.9581 | 0.9544 | 0.9830   |


### Framework versions

- Transformers 4.39.1
- Pytorch 2.2.0a0+6a974be
- Datasets 2.18.0
- Tokenizers 0.15.2