---
license: mit
base_model: prajjwal1/bert-tiny
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: tiny_bert_28_hr_intents
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# tiny_bert_28_hr_intents

This model is a fine-tuned version of [prajjwal1/bert-tiny](https://huggingface.co/prajjwal1/bert-tiny) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4764
- Accuracy: 0.9314

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| No log        | 1.0   | 238   | 3.1723          | 0.1277   |
| No log        | 2.0   | 476   | 2.9582          | 0.3735   |
| 3.1618        | 3.0   | 714   | 2.7791          | 0.4350   |
| 3.1618        | 4.0   | 952   | 2.5973          | 0.4823   |
| 2.7996        | 5.0   | 1190  | 2.4334          | 0.5012   |
| 2.7996        | 6.0   | 1428  | 2.2703          | 0.5201   |
| 2.4391        | 7.0   | 1666  | 2.1174          | 0.5721   |
| 2.4391        | 8.0   | 1904  | 1.9785          | 0.6312   |
| 2.116         | 9.0   | 2142  | 1.8518          | 0.6501   |
| 2.116         | 10.0  | 2380  | 1.7367          | 0.6832   |
| 1.8448        | 11.0  | 2618  | 1.6282          | 0.7139   |
| 1.8448        | 12.0  | 2856  | 1.5288          | 0.7352   |
| 1.6           | 13.0  | 3094  | 1.4400          | 0.7541   |
| 1.6           | 14.0  | 3332  | 1.3565          | 0.7778   |
| 1.411         | 15.0  | 3570  | 1.2838          | 0.7896   |
| 1.411         | 16.0  | 3808  | 1.2116          | 0.8227   |
| 1.2397        | 17.0  | 4046  | 1.1513          | 0.8227   |
| 1.2397        | 18.0  | 4284  | 1.0895          | 0.8345   |
| 1.1086        | 19.0  | 4522  | 1.0400          | 0.8416   |
| 1.1086        | 20.0  | 4760  | 0.9865          | 0.8416   |
| 1.1086        | 21.0  | 4998  | 0.9386          | 0.8487   |
| 0.9884        | 22.0  | 5236  | 0.8953          | 0.8629   |
| 0.9884        | 23.0  | 5474  | 0.8556          | 0.8652   |
| 0.8911        | 24.0  | 5712  | 0.8223          | 0.8700   |
| 0.8911        | 25.0  | 5950  | 0.7884          | 0.8771   |
| 0.8159        | 26.0  | 6188  | 0.7545          | 0.8771   |
| 0.8159        | 27.0  | 6426  | 0.7267          | 0.8842   |
| 0.7358        | 28.0  | 6664  | 0.7005          | 0.8913   |
| 0.7358        | 29.0  | 6902  | 0.6762          | 0.8983   |
| 0.682         | 30.0  | 7140  | 0.6571          | 0.8960   |
| 0.682         | 31.0  | 7378  | 0.6334          | 0.9007   |
| 0.6252        | 32.0  | 7616  | 0.6134          | 0.9078   |
| 0.6252        | 33.0  | 7854  | 0.5990          | 0.9054   |
| 0.5864        | 34.0  | 8092  | 0.5827          | 0.9078   |
| 0.5864        | 35.0  | 8330  | 0.5656          | 0.9149   |
| 0.5579        | 36.0  | 8568  | 0.5542          | 0.9125   |
| 0.5579        | 37.0  | 8806  | 0.5436          | 0.9125   |
| 0.525         | 38.0  | 9044  | 0.5319          | 0.9173   |
| 0.525         | 39.0  | 9282  | 0.5221          | 0.9220   |
| 0.5001        | 40.0  | 9520  | 0.5143          | 0.9243   |
| 0.5001        | 41.0  | 9758  | 0.5067          | 0.9220   |
| 0.5001        | 42.0  | 9996  | 0.5007          | 0.9267   |
| 0.4829        | 43.0  | 10234 | 0.4953          | 0.9267   |
| 0.4829        | 44.0  | 10472 | 0.4897          | 0.9314   |
| 0.4728        | 45.0  | 10710 | 0.4853          | 0.9314   |
| 0.4728        | 46.0  | 10948 | 0.4832          | 0.9267   |
| 0.4536        | 47.0  | 11186 | 0.4801          | 0.9291   |
| 0.4536        | 48.0  | 11424 | 0.4777          | 0.9291   |
| 0.4576        | 49.0  | 11662 | 0.4769          | 0.9291   |
| 0.4576        | 50.0  | 11900 | 0.4764          | 0.9314   |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3