tjbert-demo

This model is a fine-tuned version of google-bert/bert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5532

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.3322	200	2.1951
No log	0.6645	400	2.1004
No log	0.9967	600	1.9825
No log	1.3289	800	1.9472
No log	1.6611	1000	1.8623
No log	1.9934	1200	1.8167
No log	2.3256	1400	1.7663
No log	2.6578	1600	1.7273
No log	2.9900	1800	1.7117
No log	3.3223	2000	1.6747
No log	3.6545	2200	1.6545
No log	3.9867	2400	1.6191
No log	4.3189	2600	1.5865
No log	4.6512	2800	1.5793
No log	4.9834	3000	1.5532

Framework versions

• Transformers 4.57.1
• Pytorch 2.11.0+cu130
• Datasets 4.0.0
• Tokenizers 0.22.1