bert-finetuned-pos

This model is a fine-tuned version of bert-base-cased on the conll2003 dataset.

Model description

"This model is used for POS token classification and utilizes the CoNLL-2003 dataset. In processing this model, we used dynamic padding.

Intended uses & limitations

This model can be used for token classification in the field of pos. POS tags" or Part-of-Speech tags are labels that indicate the grammatical role of each word in a sentence (such as noun, verb, adjective, etc.). In a token classification task, each word (token) is individually analyzed to assign the appropriate POS tag. This helps the model better understand sentence structure and improves performance in tasks like translation, summarization, and information extraction

Training and evaluation data

conll2003

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

global_step=5268, training_loss=0.08724395379752214, metrics={'train_runtime': 566.4878, 'train_samples_per_second': 74.358, 'train_steps_per_second': 9.299, 'total_flos': 921087900564942.0, 'train_loss': 0.08724395379752214, 'epoch': 3.0}

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.1
• Tokenizers 0.21.1