pszemraj/bert-L-4_H-128_A-2-OCR-quality

bert_uncased_L-4_H-128_A-2-OCR-quality-classification-cls

This model is a fine-tuned version of google/bert_uncased_L-4_H-128_A-2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0422
• Accuracy: 0.99
• Num Input Tokens Seen: 57341952

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.99) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Input Tokens Seen
0.1123	0.2660	250	0.1202	0.974	8192000
0.072	0.5321	500	0.0665	0.986	16384000
0.0404	0.7981	750	0.0464	0.988	24576000
0.0255	1.0641	1000	0.0428	0.99	32765952
0.0253	1.3301	1250	0.0357	0.99	40957952
0.0329	1.5962	1500	0.0438	0.986	49149952
0.0435	1.8622	1750	0.0422	0.99	57341952

Framework versions

• Transformers 4.40.2
• Pytorch 2.2.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1