tjbert-e-demo

This model is a fine-tuned version of ania3000/untrained-tjbert-e on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.2437

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.6116	200	6.0650
No log	1.2232	400	5.8033
No log	1.8349	600	5.6377
No log	2.4465	800	5.5243
No log	3.0581	1000	5.4072
No log	3.6697	1200	5.3189
No log	4.2813	1400	5.3018
No log	4.8930	1600	5.2437

Framework versions

• Transformers 4.57.1
• Pytorch 2.11.0+cu130
• Datasets 4.0.0
• Tokenizers 0.22.1