Edit model card

b2b_paraphrase_retrain

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 3.0760

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 32
  • total_train_batch_size: 512
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 50
  • training_steps: 200
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.7654 5.0 5 10.5282
0.612 10.0 10 7.3206
0.4323 15.0 15 6.1823
0.3377 20.0 20 4.8119
0.2952 25.0 25 4.4300
0.2734 30.0 30 4.2043
0.2613 35.0 35 4.0847
0.2537 40.0 40 3.9372
0.248 45.0 45 3.8550
0.2429 50.0 50 3.7778
0.2392 55.0 55 3.7219
0.2352 60.0 60 3.6906
0.2316 65.0 65 3.6539
0.2275 70.0 70 3.5842
0.2242 75.0 75 3.5152
0.2197 80.0 80 3.4511
0.2155 85.0 85 3.3881
0.211 90.0 90 3.3397
0.2061 95.0 95 3.2789
0.2013 100.0 100 3.2271
0.1967 105.0 105 3.1762
0.1915 110.0 110 3.1301
0.1868 115.0 115 3.1049
0.1829 120.0 120 3.0994
0.1784 125.0 125 3.0864
0.1743 130.0 130 3.0954
0.1709 135.0 135 3.0665
0.1683 140.0 140 3.0645
0.1637 145.0 145 3.0568
0.1614 150.0 150 3.0681
0.1573 155.0 155 3.0760

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.2
  • Tokenizers 0.19.1
Downloads last month
5
Safetensors
Model size
247M params
Tensor type
F32
·
Inference API
This model can be loaded on Inference API (serverless).