moonshine_tiny_pt_v04

This model is a fine-tuned version of aomocelin/moonshine_tiny_pt_v03 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 12.6283
• Wer: 10.6467

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.03
• training_steps: 15000
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.6870	0.0529	100	12.4651	15.7720
1.7028	0.1058	200	12.4821	15.8487
1.6909	0.1587	300	12.4669	16.1938
1.7163	0.2116	400	12.4997	16.0915
1.7061	0.2646	500	12.4547	15.6697
1.7273	0.3175	600	12.4157	15.4780
1.7111	0.3704	700	12.4233	15.6058
1.7029	0.4233	800	12.4589	15.7081
1.7138	0.4762	900	12.4731	15.1840
1.7081	0.5291	1000	12.4076	15.0562
1.7061	0.5820	1100	12.4796	15.2735
1.7060	0.6349	1200	12.4509	15.0435
1.6852	0.6878	1300	12.4578	15.0690
1.7216	0.7407	1400	12.4481	14.9156
1.6963	0.7937	1500	12.5131	14.5706
1.7025	0.8466	1600	12.5256	14.7111
1.7095	0.8995	1700	12.4896	14.5322
1.6989	0.9524	1800	12.4903	14.5322
1.6933	1.0053	1900	12.4363	14.4811
1.6478	1.0582	2000	12.4675	14.5450
1.6627	1.1111	2100	12.5061	14.4683
1.6610	1.1640	2200	12.4718	14.3405
1.6579	1.2169	2300	12.5381	14.5450
1.6633	1.2698	2400	12.4929	14.4044
1.6956	1.3228	2500	12.5084	13.9443
1.6695	1.3757	2600	12.5300	14.1871
1.6847	1.4286	2700	12.5144	14.1871
1.6490	1.4815	2800	12.4924	13.9826
1.6667	1.5344	2900	12.5115	13.7909
1.6628	1.5873	3000	12.5341	13.9059
1.6793	1.6402	3100	12.5363	13.6503
1.6616	1.6931	3200	12.4976	13.8676
1.6500	1.7460	3300	12.5131	13.7014
1.6461	1.7989	3400	12.5559	13.8165
1.6462	1.8519	3500	12.5579	13.7909
1.6538	1.9048	3600	12.5676	13.7142
1.6646	1.9577	3700	12.5716	13.4458
1.6406	2.0106	3800	12.5429	13.5353
1.6341	2.0635	3900	12.5872	13.4202
1.6155	2.1164	4000	12.5478	13.3819
1.6352	2.1693	4100	12.5362	13.1391
1.6371	2.2222	4200	12.5187	13.0240
1.6527	2.2751	4300	12.5563	13.6375
1.6239	2.3280	4400	12.5762	12.9601
1.6414	2.3810	4500	12.5629	13.1135
1.6488	2.4339	4600	12.5774	13.1007
1.6348	2.4868	4700	12.5386	12.7684
1.6359	2.5397	4800	12.5342	12.4489
1.6356	2.5926	4900	12.5610	12.7556
1.6363	2.6455	5000	12.5918	12.7556
1.6227	2.6984	5100	12.5607	12.6150
1.6370	2.7513	5200	12.5384	12.5767
1.6283	2.8042	5300	12.5642	12.4233
1.6369	2.8571	5400	12.6061	12.4489
1.6348	2.9101	5500	12.5971	12.6278
1.6272	2.9630	5600	12.5752	12.3466
1.6134	3.0159	5700	12.5791	12.2572
1.6055	3.0688	5800	12.6017	12.0399
1.6050	3.1217	5900	12.5832	12.2060
1.6147	3.1746	6000	12.5757	12.1421
1.6063	3.2275	6100	12.5882	12.2444
1.6078	3.2804	6200	12.5847	12.1293
1.6088	3.3333	6300	12.5687	12.1677
1.6082	3.3862	6400	12.5613	12.2316
1.5959	3.4392	6500	12.5568	12.2316
1.6009	3.4921	6600	12.6016	11.9376
1.6224	3.5450	6700	12.5613	12.1421
1.6035	3.5979	6800	12.5800	11.7970
1.6210	3.6508	6900	12.5734	11.9888
1.6216	3.7037	7000	12.6113	11.7203
1.6025	3.7566	7100	12.5952	11.8865
1.6069	3.8095	7200	12.6191	11.9376
1.6073	3.8624	7300	12.6027	11.8737
1.6056	3.9153	7400	12.6164	11.8098
1.6044	3.9683	7500	12.6189	11.6181
1.6015	4.0212	7600	12.5824	11.7331
1.5997	4.0741	7700	12.5783	12.0271
1.6003	4.1270	7800	12.6001	11.6437
1.5855	4.1799	7900	12.5966	11.7203
1.5983	4.2328	8000	12.6151	11.5925
1.5945	4.2857	8100	12.5972	11.6692
1.6068	4.3386	8200	12.6474	11.6437
1.5990	4.3915	8300	12.5836	11.5286
1.6011	4.4444	8400	12.5895	11.3497
1.6014	4.4974	8500	12.6118	11.6820
1.6061	4.5503	8600	12.6297	11.6437
1.5863	4.6032	8700	12.6182	11.4264
1.6050	4.6561	8800	12.6215	11.2858
1.5847	4.7090	8900	12.5973	11.6437
1.5763	4.7619	9000	12.6168	11.4392
1.5781	4.8148	9100	12.6074	11.3113
1.5863	4.8677	9200	12.6096	11.3497
1.5906	4.9206	9300	12.6095	11.1708
1.5850	4.9735	9400	12.6052	11.1324
1.5737	5.0265	9500	12.6376	11.2858
1.5937	5.0794	9600	12.6046	10.9790
1.5859	5.1323	9700	12.5875	11.3369
1.5787	5.1852	9800	12.5944	11.1324
1.5763	5.2381	9900	12.6216	11.3880
1.5700	5.2910	10000	12.5924	11.1708
1.5939	5.3439	10100	12.6294	10.9790
1.5790	5.3968	10200	12.6101	11.2474
1.5787	5.4497	10300	12.6190	11.1324
1.5731	5.5026	10400	12.6185	11.1835
1.5919	5.5556	10500	12.6126	11.1835
1.5724	5.6085	10600	12.6362	10.8640
1.5831	5.6614	10700	12.6422	10.9535
1.5943	5.7143	10800	12.6476	10.8512
1.5743	5.7672	10900	12.6147	11.0046
1.5854	5.8201	11000	12.6214	11.0429
1.5784	5.8730	11100	12.6164	10.9663
1.5752	5.9259	11200	12.6222	10.8768
1.5945	5.9788	11300	12.6221	10.9151
1.5755	6.0317	11400	12.6116	10.9918
1.5623	6.0847	11500	12.6051	10.6979
1.5774	6.1376	11600	12.6166	10.8768
1.5662	6.1905	11700	12.6177	10.6467
1.5763	6.2434	11800	12.6038	10.8001
1.5710	6.2963	11900	12.6045	10.9918
1.5686	6.3492	12000	12.6176	10.7745
1.5692	6.4021	12100	12.6117	10.9790
1.5699	6.4550	12200	12.6134	10.8512
1.5678	6.5079	12300	12.6228	10.8512
1.6014	6.5608	12400	12.6069	10.8768
1.5806	6.6138	12500	12.6237	10.8512
1.5784	6.6667	12600	12.6422	10.6723
1.5766	6.7196	12700	12.6170	10.7873
1.5671	6.7725	12800	12.6274	10.4806
1.5770	6.8254	12900	12.6285	10.5956
1.5747	6.8783	13000	12.6399	10.6339
1.5729	6.9312	13100	12.6434	10.4678
1.5704	6.9841	13200	12.6178	10.6595
1.5672	7.0370	13300	12.6234	10.6084
1.5714	7.0899	13400	12.6279	10.5573
1.5670	7.1429	13500	12.6218	10.8512
1.5719	7.1958	13600	12.6283	10.7873
1.5726	7.2487	13700	12.6301	10.7745
1.5748	7.3016	13800	12.6233	10.7234
1.5703	7.3545	13900	12.6222	10.7490
1.5731	7.4074	14000	12.6300	10.5445
1.5691	7.4603	14100	12.6176	10.5445
1.5674	7.5132	14200	12.6253	10.4934
1.5709	7.5661	14300	12.6125	10.5445
1.5800	7.6190	14400	12.6193	10.5317
1.5733	7.6720	14500	12.6046	10.4550
1.5651	7.7249	14600	12.6130	10.4934
1.5628	7.7778	14700	12.6171	10.5700
1.5643	7.8307	14800	12.6255	10.5445
1.5647	7.8836	14900	12.6140	10.4934
1.5742	7.9365	15000	12.6283	10.6467

Framework versions

• Transformers 5.12.1
• Pytorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2