moonshine_tiny_pt_v06

This model is a fine-tuned version of aomocelin/moonshine_tiny_pt_v05 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 11.9592
• Wer: 2.9991

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.03
• training_steps: 15000
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.8875	0.3333	100	11.9384	13.8238
1.7425	0.6667	200	12.4432	8.8566
1.6736	1.0	300	12.1785	5.4358
1.5927	1.3333	400	12.2073	3.9831
1.5491	1.6667	500	12.1213	3.1396
1.5326	2.0	600	12.1534	2.9053
1.4935	2.3333	700	12.1736	2.5773
1.5101	2.6667	800	12.1377	2.5773
1.4717	3.0	900	12.0547	2.3430
1.4700	3.3333	1000	12.1816	2.6242
1.4857	3.6667	1100	12.2244	2.4836
1.4602	4.0	1200	12.0599	2.3899
1.4901	4.3333	1300	12.2100	2.2024
1.4568	4.6667	1400	12.1898	2.3430
1.4449	5.0	1500	12.0739	2.2962
1.4465	5.3333	1600	12.1867	2.2962
1.4538	5.6667	1700	12.0917	2.2024
1.4551	6.0	1800	12.1330	2.3899
1.4373	6.3333	1900	12.0774	2.4367
1.4358	6.6667	2000	12.0647	2.2493
1.4357	7.0	2100	12.1909	2.2493
1.4326	7.3333	2200	12.1128	2.4836
1.4315	7.6667	2300	12.1478	2.3899
1.4385	8.0	2400	12.0244	2.4367
1.4237	8.3333	2500	12.0461	2.2493
1.4285	8.6667	2600	12.1703	2.5305
1.4251	9.0	2700	12.1301	2.2024
1.4206	9.3333	2800	12.1147	2.2962
1.4189	9.6667	2900	12.0485	2.1556
1.4258	10.0	3000	12.1007	2.3899
1.4206	10.3333	3100	12.1193	2.2024
1.4148	10.6667	3200	11.9768	2.4836
1.4277	11.0	3300	12.1122	2.3899
1.4144	11.3333	3400	12.1137	2.3899
1.4203	11.6667	3500	12.1734	2.1556
1.4178	12.0	3600	12.2454	2.4836
1.4139	12.3333	3700	12.2105	2.4367
1.4140	12.6667	3800	12.0925	2.4836
1.4175	13.0	3900	12.0842	2.3899
1.4134	13.3333	4000	12.1275	2.5305
1.4119	13.6667	4100	12.0579	2.6242
1.4150	14.0	4200	12.1589	2.6242
1.4109	14.3333	4300	12.1373	2.4836
1.4101	14.6667	4400	12.0742	2.3899
1.4170	15.0	4500	12.1680	2.4836
1.4118	15.3333	4600	11.9749	2.5305
1.4098	15.6667	4700	12.0181	2.6242
1.4119	16.0	4800	11.9903	2.6242
1.4118	16.3333	4900	12.0611	2.4367
1.4128	16.6667	5000	12.0391	2.5773
1.4083	17.0	5100	12.1308	2.6242
1.4085	17.3333	5200	12.1002	2.6242
1.4089	17.6667	5300	12.0349	2.5773
1.4109	18.0	5400	12.0256	2.7179
1.4071	18.3333	5500	12.0432	2.7179
1.4089	18.6667	5600	12.0091	2.6710
1.4055	19.0	5700	12.1616	2.4836
1.4069	19.3333	5800	12.1005	2.8116
1.4047	19.6667	5900	12.0644	2.8585
1.4094	20.0	6000	11.9779	2.7648
1.4070	20.3333	6100	12.0492	2.7648
1.4063	20.6667	6200	12.1331	2.9053
1.4050	21.0	6300	12.0253	2.8116
1.4049	21.3333	6400	12.0808	2.9991
1.4059	21.6667	6500	12.1929	2.8585
1.4040	22.0	6600	12.0342	2.7179
1.4047	22.3333	6700	11.9977	2.7648
1.4040	22.6667	6800	11.8829	3.1396
1.4040	23.0	6900	11.9778	2.9991
1.4038	23.3333	7000	11.9951	2.9053
1.4031	23.6667	7100	11.9650	2.9053
1.4020	24.0	7200	12.0531	2.7648
1.4008	24.3333	7300	12.0262	2.8116
1.4018	24.6667	7400	12.0307	2.9522
1.4037	25.0	7500	11.9964	2.9522
1.4018	25.3333	7600	12.0413	2.9991
1.4024	25.6667	7700	11.9425	2.9522
1.4024	26.0	7800	12.0068	2.8585
1.4022	26.3333	7900	12.0230	2.8116
1.4012	26.6667	8000	12.0530	2.9522
1.4017	27.0	8100	12.0759	2.9522
1.4015	27.3333	8200	12.0123	2.9522
1.4005	27.6667	8300	12.0471	2.7648
1.4013	28.0	8400	11.9650	2.9522
1.4018	28.3333	8500	11.9206	2.8585
1.4002	28.6667	8600	12.0343	2.9053
1.4002	29.0	8700	12.0737	2.8585
1.4011	29.3333	8800	12.1040	2.8585
1.3988	29.6667	8900	12.0688	2.9522
1.4004	30.0	9000	12.0805	2.8116
1.3999	30.3333	9100	12.0540	2.7648
1.3995	30.6667	9200	12.0245	2.9053
1.3995	31.0	9300	12.0017	2.9053
1.3981	31.3333	9400	12.0664	2.8585
1.3987	31.6667	9500	11.9616	2.8585
1.4008	32.0	9600	11.9159	2.8116
1.3982	32.3333	9700	12.0014	2.9053
1.3995	32.6667	9800	11.9899	2.9053
1.3998	33.0	9900	11.8816	2.9522
1.3981	33.3333	10000	12.0703	2.8585
1.3987	33.6667	10100	12.0297	2.8585
1.3977	34.0	10200	12.0417	2.7648
1.3983	34.3333	10300	11.9480	2.9522
1.3983	34.6667	10400	11.9015	2.8116
1.3985	35.0	10500	11.9455	2.9991
1.3974	35.3333	10600	11.9939	2.9053
1.3965	35.6667	10700	12.0638	2.9991
1.3991	36.0	10800	11.9777	2.9053
1.3972	36.3333	10900	12.0333	2.8585
1.3967	36.6667	11000	11.9579	2.8585
1.3987	37.0	11100	11.9382	2.9522
1.3991	37.3333	11200	12.0107	2.9053
1.3965	37.6667	11300	11.8221	3.1396
1.3959	38.0	11400	12.0086	2.8585
1.3971	38.3333	11500	11.9671	2.8585
1.3966	38.6667	11600	12.0201	2.9991
1.3969	39.0	11700	12.0182	2.8585
1.3971	39.3333	11800	11.9672	3.2802
1.3955	39.6667	11900	11.9890	2.9991
1.3973	40.0	12000	12.0208	2.8585
1.3963	40.3333	12100	12.0048	3.1396
1.3964	40.6667	12200	11.9409	3.2802
1.3967	41.0	12300	11.9929	2.9991
1.3974	41.3333	12400	11.9783	2.9053
1.3959	41.6667	12500	12.1101	2.9522
1.3977	42.0	12600	11.9839	2.9522
1.3965	42.3333	12700	11.8602	3.2802
1.3953	42.6667	12800	11.9893	3.1865
1.3959	43.0	12900	11.8309	3.2802
1.3962	43.3333	13000	12.1124	2.9522
1.3951	43.6667	13100	12.0331	2.9522
1.3957	44.0	13200	12.0505	3.0928
1.3948	44.3333	13300	12.0738	3.0459
1.3969	44.6667	13400	12.0561	3.0459
1.3955	45.0	13500	11.9294	3.1865
1.3950	45.3333	13600	12.0358	3.2334
1.3957	45.6667	13700	12.0231	2.9991
1.3953	46.0	13800	11.9452	3.1865
1.3956	46.3333	13900	12.0800	3.1396
1.3959	46.6667	14000	11.9762	3.1865
1.3960	47.0	14100	11.9765	3.2334
1.3963	47.3333	14200	11.9792	3.1865
1.3938	47.6667	14300	11.9729	3.1396
1.3964	48.0	14400	11.9802	3.0459
1.3948	48.3333	14500	12.0234	2.9522
1.3957	48.6667	14600	11.9746	3.1396
1.3975	49.0	14700	12.1101	2.9522
1.3948	49.3333	14800	12.0146	2.9991
1.3963	49.6667	14900	11.9949	2.9522
1.3957	50.0	15000	11.9592	2.9991

Framework versions

• Transformers 5.12.1
• Pytorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2