Visualize in Weights & Biases

exceptions_exp2_swap_0.3_resemble_to_push_2128

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 3.5628
  • Accuracy: 0.3689

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0006
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 2128
  • gradient_accumulation_steps: 5
  • total_train_batch_size: 80
  • optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 50.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
4.8109 0.2915 1000 4.7588 0.2545
4.3409 0.5830 2000 4.2851 0.2990
4.1487 0.8745 3000 4.0986 0.3151
3.9935 1.1659 4000 3.9946 0.3244
3.9482 1.4574 5000 3.9187 0.3308
3.88 1.7488 6000 3.8616 0.3362
3.7453 2.0402 7000 3.8210 0.3407
3.7605 2.3317 8000 3.7884 0.3437
3.7404 2.6232 9000 3.7612 0.3464
3.7262 2.9147 10000 3.7332 0.3487
3.6415 3.2061 11000 3.7201 0.3506
3.6534 3.4976 12000 3.7031 0.3523
3.6462 3.7891 13000 3.6828 0.3539
3.5516 4.0805 14000 3.6781 0.3553
3.5821 4.3719 15000 3.6676 0.3564
3.568 4.6634 16000 3.6518 0.3578
3.5882 4.9549 17000 3.6409 0.3585
3.4946 5.2463 18000 3.6418 0.3591
3.533 5.5378 19000 3.6313 0.3602
3.5232 5.8293 20000 3.6190 0.3613
3.4486 6.1207 21000 3.6250 0.3615
3.4855 6.4122 22000 3.6169 0.3620
3.4814 6.7037 23000 3.6053 0.3629
3.5027 6.9952 24000 3.5974 0.3637
3.4279 7.2865 25000 3.6066 0.3637
3.4646 7.5780 26000 3.5969 0.3642
3.459 7.8695 27000 3.5860 0.3652
3.3867 8.1609 28000 3.5984 0.3645
3.433 8.4524 29000 3.5886 0.3655
3.433 8.7439 30000 3.5821 0.3661
3.3437 9.0353 31000 3.5859 0.3662
3.3821 9.3268 32000 3.5854 0.3663
3.4138 9.6183 33000 3.5769 0.3668
3.4159 9.9098 34000 3.5669 0.3676
3.3383 10.2011 35000 3.5799 0.3674
3.3616 10.4926 36000 3.5728 0.3676
3.3799 10.7841 37000 3.5630 0.3683
3.2901 11.0755 38000 3.5756 0.3678
3.3376 11.3670 39000 3.5725 0.3678
3.3722 11.6585 40000 3.5628 0.3689
3.3741 11.9500 41000 3.5549 0.3694
3.3114 12.2414 42000 3.5692 0.3689
3.3512 12.5329 43000 3.5621 0.3693
3.3496 12.8243 44000 3.5528 0.3700
3.268 13.1157 45000 3.5699 0.3692
3.3054 13.4072 46000 3.5612 0.3696
3.3365 13.6987 47000 3.5542 0.3701
3.3383 13.9902 48000 3.5467 0.3708
3.2765 14.2816 49000 3.5612 0.3702
3.3014 14.5731 50000 3.5549 0.3706
3.3378 14.8646 51000 3.5469 0.3710
3.2513 15.1559 52000 3.5629 0.3703
3.2827 15.4474 53000 3.5542 0.3710
3.3096 15.7389 54000 3.5491 0.3712
3.2072 16.0303 55000 3.5587 0.3707
3.2618 16.3218 56000 3.5567 0.3710
3.2877 16.6133 57000 3.5498 0.3713
3.306 16.9048 58000 3.5405 0.3721
3.2393 17.1962 59000 3.5580 0.3710
3.2617 17.4877 60000 3.5486 0.3718
3.2826 17.7792 61000 3.5424 0.3722
3.2007 18.0705 62000 3.5563 0.3716
3.2398 18.3620 63000 3.5540 0.3714
3.2587 18.6535 64000 3.5450 0.3721
3.2839 18.9450 65000 3.5357 0.3728
3.2095 19.2364 66000 3.5559 0.3719
3.2479 19.5279 67000 3.5470 0.3725
3.271 19.8194 68000 3.5406 0.3728
3.1877 20.1108 69000 3.5589 0.3720
3.2304 20.4023 70000 3.5502 0.3723
3.2535 20.6938 71000 3.5434 0.3728
3.2638 20.9853 72000 3.5362 0.3733
3.2027 21.2766 73000 3.5519 0.3726
3.2296 21.5681 74000 3.5439 0.3730
3.2565 21.8596 75000 3.5351 0.3734
3.1675 22.1510 76000 3.5585 0.3726
3.2115 22.4425 77000 3.5471 0.3729
3.2371 22.7340 78000 3.5407 0.3731
3.1285 23.0254 79000 3.5514 0.3730
3.188 23.3169 80000 3.5513 0.3726
3.2126 23.6083 81000 3.5469 0.3730
3.2242 23.8998 82000 3.5355 0.3737
3.1645 24.1912 83000 3.5555 0.3733
3.1905 24.4827 84000 3.5489 0.3735
3.2129 24.7742 85000 3.5382 0.3737
3.1326 25.0656 86000 3.5581 0.3730
3.1723 25.3571 87000 3.5494 0.3735
3.1961 25.6486 88000 3.5412 0.3740
3.2203 25.9401 89000 3.5363 0.3741
3.1438 26.2314 90000 3.5543 0.3732
3.1887 26.5229 91000 3.5443 0.3737
3.1887 26.8144 92000 3.5406 0.3741
3.123 27.1058 93000 3.5566 0.3734
3.1626 27.3973 94000 3.5523 0.3735
3.178 27.6888 95000 3.5431 0.3739

Framework versions

  • Transformers 4.55.2
  • Pytorch 2.8.0+cu128
  • Datasets 4.0.0
  • Tokenizers 0.21.4
Downloads last month
11
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support