100M_low_500_495

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.3007
• Accuracy: 0.3945

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0006
• train_batch_size: 32
• eval_batch_size: 16
• seed: 495
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.98) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
5.0925	0.1078	1000	5.0207	0.2273
4.5826	0.2156	2000	4.5255	0.2688
4.3361	0.3235	3000	4.2429	0.2974
4.1659	0.4313	4000	4.0975	0.3122
4.0552	0.5391	5000	3.9984	0.3209
4.0014	0.6469	6000	3.9248	0.3272
3.9359	0.7547	7000	3.8724	0.3323
3.8791	0.8625	8000	3.8246	0.3367
3.8491	0.9704	9000	3.7871	0.3408
3.7685	1.0782	10000	3.7571	0.3436
3.7832	1.1860	11000	3.7309	0.3458
3.7448	1.2938	12000	3.7076	0.3478
3.717	1.4016	13000	3.6833	0.3507
3.7042	1.5094	14000	3.6630	0.3525
3.693	1.6173	15000	3.6447	0.3545
3.6681	1.7251	16000	3.6263	0.3563
3.6691	1.8329	17000	3.6127	0.3578
3.6447	1.9407	18000	3.5970	0.3588
3.587	2.0485	19000	3.5873	0.3606
3.5655	2.1563	20000	3.5788	0.3617
3.5495	2.2642	21000	3.5679	0.3628
3.5554	2.3720	22000	3.5587	0.3641
3.5481	2.4798	23000	3.5474	0.3650
3.5407	2.5876	24000	3.5363	0.3661
3.5515	2.6954	25000	3.5235	0.3672
3.5534	2.8032	26000	3.5183	0.3678
3.5398	2.9111	27000	3.5073	0.3690
3.4382	3.0189	28000	3.5020	0.3699
3.454	3.1267	29000	3.4990	0.3705
3.472	3.2345	30000	3.4921	0.3709
3.4586	3.3423	31000	3.4878	0.3720
3.4665	3.4501	32000	3.4812	0.3722
3.4546	3.5580	33000	3.4747	0.3731
3.4782	3.6658	34000	3.4664	0.3738
3.4543	3.7736	35000	3.4612	0.3742
3.4392	3.8814	36000	3.4534	0.3750
3.4613	3.9892	37000	3.4493	0.3755
3.3757	4.0970	38000	3.4521	0.3764
3.3854	4.2049	39000	3.4457	0.3765
3.3893	4.3127	40000	3.4418	0.3771
3.4053	4.4205	41000	3.4355	0.3776
3.4045	4.5283	42000	3.4294	0.3784
3.3941	4.6361	43000	3.4254	0.3783
3.4	4.7439	44000	3.4218	0.3793
3.403	4.8518	45000	3.4149	0.3797
3.3859	4.9596	46000	3.4115	0.3803
3.3109	5.0674	47000	3.4175	0.3803
3.3138	5.1752	48000	3.4124	0.3808
3.3378	5.2830	49000	3.4071	0.3812
3.3421	5.3908	50000	3.4068	0.3815
3.343	5.4987	51000	3.4004	0.3821
3.3504	5.6065	52000	3.3944	0.3825
3.3439	5.7143	53000	3.3909	0.3827
3.3313	5.8221	54000	3.3855	0.3834
3.3531	5.9299	55000	3.3823	0.3837
3.2507	6.0377	56000	3.3856	0.3840
3.2697	6.1456	57000	3.3823	0.3840
3.2864	6.2534	58000	3.3805	0.3845
3.2703	6.3612	59000	3.3770	0.3850
3.2881	6.4690	60000	3.3736	0.3847
3.2802	6.5768	61000	3.3686	0.3856
3.3016	6.6846	62000	3.3637	0.3863
3.274	6.7925	63000	3.3623	0.3863
3.2783	6.9003	64000	3.3567	0.3869
3.2099	7.0081	65000	3.3589	0.3873
3.2031	7.1159	66000	3.3606	0.3869
3.2278	7.2237	67000	3.3592	0.3875
3.2282	7.3315	68000	3.3537	0.3877
3.2237	7.4394	69000	3.3503	0.3878
3.2348	7.5472	70000	3.3463	0.3884
3.2284	7.6550	71000	3.3422	0.3889
3.233	7.7628	72000	3.3397	0.3894
3.2421	7.8706	73000	3.3369	0.3895
3.2536	7.9784	74000	3.3328	0.3902
3.1647	8.0863	75000	3.3391	0.3898
3.1876	8.1941	76000	3.3366	0.3902
3.1866	8.3019	77000	3.3340	0.3905
3.1783	8.4097	78000	3.3318	0.3907
3.1778	8.5175	79000	3.3285	0.3912
3.2113	8.6253	80000	3.3226	0.3916
3.1655	8.7332	81000	3.3209	0.3917
3.1812	8.8410	82000	3.3189	0.3921
3.1955	8.9488	83000	3.3142	0.3926
3.132	9.0566	84000	3.3179	0.3925
3.1264	9.1644	85000	3.3165	0.3927
3.1268	9.2722	86000	3.3139	0.3930
3.13	9.3801	87000	3.3120	0.3932
3.1275	9.4879	88000	3.3094	0.3936
3.1298	9.5957	89000	3.3061	0.3938
3.1371	9.7035	90000	3.3039	0.3942
3.1463	9.8113	91000	3.3014	0.3944
3.1472	9.9191	92000	3.3007	0.3945

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.5.0+cu124
• Datasets 3.0.2
• Tokenizers 0.20.1