moonshine_tiny_pt_v08

This model is a fine-tuned version of aomocelin/moonshine_tiny_pt_v05 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4744
• Wer: 5.8564

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.03
• training_steps: 15000
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.9572	0.3333	100	0.6952	18.0730
1.8258	0.6667	200	0.5315	15.4912
1.7541	1.0	300	0.4986	10.7683
1.6868	1.3333	400	0.4044	9.1310
1.6393	1.6667	500	0.3660	7.6196
1.6245	2.0	600	0.3996	7.3048
1.6085	2.3333	700	0.3359	6.5491
1.5690	2.6667	800	0.3037	6.5491
1.5405	3.0	900	0.3275	6.5491
1.5349	3.3333	1000	0.2825	6.7380
1.5689	3.6667	1100	0.2905	6.4861
1.5107	4.0	1200	0.3599	6.1083
1.5289	4.3333	1300	0.2912	6.1713
1.5025	4.6667	1400	0.2863	5.9194
1.5333	5.0	1500	0.2940	5.6045
1.4817	5.3333	1600	0.3379	5.9824
1.4951	5.6667	1700	0.3404	5.7305
1.4967	6.0	1800	0.2571	5.7935
1.4816	6.3333	1900	0.3554	5.5416
1.4883	6.6667	2000	0.2896	5.8564
1.4787	7.0	2100	0.2910	5.4156
1.4678	7.3333	2200	0.2979	5.1637
1.4763	7.6667	2300	0.3495	5.5416
1.4773	8.0	2400	0.2755	5.6045
1.4460	8.3333	2500	0.3962	5.4156
1.4775	8.6667	2600	0.2898	5.1637
1.4471	9.0	2700	0.4108	5.2897
1.4483	9.3333	2800	0.3196	5.1008
1.4598	9.6667	2900	0.2827	5.4156
1.4554	10.0	3000	0.2967	4.9118
1.4467	10.3333	3100	0.3497	4.9748
1.4623	10.6667	3200	0.2960	5.4156
1.4476	11.0	3300	0.3147	5.0378
1.4450	11.3333	3400	0.3258	5.2267
1.4391	11.6667	3500	0.3323	5.4156
1.4456	12.0	3600	0.3375	4.7859
1.4365	12.3333	3700	0.3142	5.3526
1.4485	12.6667	3800	0.3365	5.2897
1.4340	13.0	3900	0.2697	5.0378
1.4367	13.3333	4000	0.3053	5.6045
1.4343	13.6667	4100	0.4018	5.5416
1.4331	14.0	4200	0.3114	5.1008
1.4355	14.3333	4300	0.2924	5.2267
1.4367	14.6667	4400	0.3988	5.2897
1.4304	15.0	4500	0.3433	5.3526
1.4310	15.3333	4600	0.3360	5.2897
1.4303	15.6667	4700	0.2975	5.6045
1.4295	16.0	4800	0.3129	5.2267
1.4308	16.3333	4900	0.3739	5.4786
1.4292	16.6667	5000	0.3630	5.2897
1.4296	17.0	5100	0.4181	5.4786
1.4247	17.3333	5200	0.3273	5.4156
1.4247	17.6667	5300	0.3452	5.4156
1.4264	18.0	5400	0.3196	5.3526
1.4254	18.3333	5500	0.3364	5.2897
1.4256	18.6667	5600	0.3602	5.2897
1.4239	19.0	5700	0.3918	5.2897
1.4256	19.3333	5800	0.3567	5.2897
1.4204	19.6667	5900	0.3675	5.4786
1.4229	20.0	6000	0.3502	5.6045
1.4216	20.3333	6100	0.3632	5.2267
1.4234	20.6667	6200	0.3388	5.6045
1.4236	21.0	6300	0.3506	5.4786
1.4211	21.3333	6400	0.2847	5.6675
1.4186	21.6667	6500	0.3534	5.7305
1.4204	22.0	6600	0.3801	5.5416
1.4196	22.3333	6700	0.3071	5.3526
1.4197	22.6667	6800	0.4321	5.9194
1.4207	23.0	6900	0.3133	5.6045
1.4173	23.3333	7000	0.3872	5.6045
1.4198	23.6667	7100	0.3663	5.6675
1.4197	24.0	7200	0.3305	5.5416
1.4184	24.3333	7300	0.3395	5.6045
1.4174	24.6667	7400	0.4112	5.6675
1.4179	25.0	7500	0.3492	5.6675
1.4163	25.3333	7600	0.3480	5.5416
1.4153	25.6667	7700	0.3893	5.8564
1.4180	26.0	7800	0.2934	5.4786
1.4196	26.3333	7900	0.3670	5.7935
1.4143	26.6667	8000	0.3728	5.7935
1.4148	27.0	8100	0.2934	5.7305
1.4173	27.3333	8200	0.3474	5.7305
1.4137	27.6667	8300	0.3719	5.7305
1.4151	28.0	8400	0.3645	5.6675
1.4139	28.3333	8500	0.3531	5.4786
1.4158	28.6667	8600	0.4046	5.7305
1.4155	29.0	8700	0.3693	5.8564
1.4167	29.3333	8800	0.3415	5.2897
1.4148	29.6667	8900	0.3808	5.9194
1.4120	30.0	9000	0.3218	5.9824
1.4138	30.3333	9100	0.3731	5.7305
1.4136	30.6667	9200	0.3531	6.1083
1.4138	31.0	9300	0.4045	5.7935
1.4113	31.3333	9400	0.3701	5.9194
1.4126	31.6667	9500	0.3456	6.1713
1.4122	32.0	9600	0.3767	5.5416
1.4134	32.3333	9700	0.5179	5.8564
1.4121	32.6667	9800	0.3466	5.8564
1.4131	33.0	9900	0.3928	5.9194
1.4108	33.3333	10000	0.3777	5.9824
1.4117	33.6667	10100	0.4173	5.7305
1.4106	34.0	10200	0.3459	6.0453
1.4129	34.3333	10300	0.3187	5.9194
1.4103	34.6667	10400	0.3936	5.9194
1.4103	35.0	10500	0.3473	5.9194
1.4120	35.3333	10600	0.3499	6.1083
1.4113	35.6667	10700	0.3672	5.9824
1.4108	36.0	10800	0.3814	6.0453
1.4115	36.3333	10900	0.3811	5.9824
1.4093	36.6667	11000	0.3922	6.0453
1.4105	37.0	11100	0.3137	6.1083
1.4101	37.3333	11200	0.3293	6.1083
1.4106	37.6667	11300	0.3699	5.9824
1.4107	38.0	11400	0.3415	5.8564
1.4102	38.3333	11500	0.3366	5.7935
1.4102	38.6667	11600	0.3566	6.1083
1.4099	39.0	11700	0.3265	6.0453
1.4089	39.3333	11800	0.4344	5.8564
1.4085	39.6667	11900	0.3473	6.1713
1.4100	40.0	12000	0.3714	5.7935
1.4084	40.3333	12100	0.4120	5.8564
1.4097	40.6667	12200	0.3376	6.0453
1.4092	41.0	12300	0.4747	6.1083
1.4077	41.3333	12400	0.4180	6.0453
1.4087	41.6667	12500	0.3869	6.0453
1.4092	42.0	12600	0.4347	6.0453
1.4075	42.3333	12700	0.4189	6.0453
1.4084	42.6667	12800	0.4041	6.0453
1.4080	43.0	12900	0.3731	6.0453
1.4056	43.3333	13000	0.4065	6.0453
1.4088	43.6667	13100	0.4018	5.8564
1.4083	44.0	13200	0.4215	6.1083
1.4071	44.3333	13300	0.3670	6.0453
1.4077	44.6667	13400	0.4417	5.9824
1.4101	45.0	13500	0.3986	6.1083
1.4079	45.3333	13600	0.3619	6.1713
1.4065	45.6667	13700	0.4336	6.0453
1.4079	46.0	13800	0.4404	6.1713
1.4077	46.3333	13900	0.3950	5.9824
1.4081	46.6667	14000	0.4177	6.0453
1.4081	47.0	14100	0.4008	6.0453
1.4073	47.3333	14200	0.3697	6.1083
1.4085	47.6667	14300	0.3472	6.0453
1.4090	48.0	14400	0.4068	6.0453
1.4067	48.3333	14500	0.3894	6.0453
1.4074	48.6667	14600	0.3613	6.1713
1.4083	49.0	14700	0.3718	6.1083
1.4086	49.3333	14800	0.4073	6.2343
1.4074	49.6667	14900	0.4400	6.1083
1.4075	50.0	15000	0.4744	5.8564

Framework versions

• Transformers 5.13.0
• Pytorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2