moonshine_tiny_pt_v03

This model is a fine-tuned version of aomocelin/moonshine_tiny_pt_v02 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 12.5181
• Wer: 16.0510

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.03
• training_steps: 15000
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.1458	0.0617	100	11.7179	32.4170
2.1220	0.1235	200	11.8265	31.9015
2.1657	0.1852	300	11.8987	31.0424
2.0808	0.2469	400	11.9196	30.2692
2.0488	0.3086	500	11.9262	30.3408
2.0902	0.3704	600	11.8829	29.5819
2.0569	0.4321	700	11.8932	29.0235
2.0674	0.4938	800	11.9391	28.9662
2.0154	0.5556	900	11.9135	28.5653
2.0080	0.6173	1000	11.9578	28.5510
2.0548	0.6790	1100	11.9317	28.0785
2.0024	0.7407	1200	11.9502	27.7778
2.0140	0.8025	1300	11.9405	27.7778
2.0105	0.8642	1400	11.9626	27.1478
2.0247	0.9259	1500	12.0054	27.4771
2.0272	0.9877	1600	11.9801	26.9187
1.9414	1.0494	1700	12.0169	26.5321
1.9549	1.1111	1800	12.0433	26.6323
1.9835	1.1728	1900	12.0334	26.1025
1.9473	1.2346	2000	12.0364	25.3723
1.9637	1.2963	2100	12.0018	25.4296
1.9764	1.3580	2200	12.0584	25.2434
1.9262	1.4198	2300	12.0248	25.2148
1.9289	1.4815	2400	12.0656	24.8711
1.9507	1.5432	2500	12.0755	25.0286
1.9316	1.6049	2600	12.0708	24.4273
1.9552	1.6667	2700	12.0675	23.9548
1.8980	1.7284	2800	12.0865	24.2984
1.9261	1.7901	2900	12.1010	23.8259
1.9401	1.8519	3000	12.0872	23.7829
1.9265	1.9136	3100	12.0965	23.2388
1.9129	1.9753	3200	12.1178	23.5109
1.8629	2.0370	3300	12.1496	23.4393
1.8972	2.0988	3400	12.1787	22.9238
1.9106	2.1605	3500	12.1319	22.9381
1.8642	2.2222	3600	12.1316	23.0241
1.8587	2.2840	3700	12.2238	22.6804
1.8739	2.3457	3800	12.2172	22.6804
1.8967	2.4074	3900	12.2015	22.8379
1.8421	2.4691	4000	12.2039	22.2652
1.8502	2.5309	4100	12.1918	22.4513
1.8473	2.5926	4200	12.1852	21.8786
1.8683	2.6543	4300	12.2736	22.2652
1.8238	2.7160	4400	12.2409	21.8070
1.8492	2.7778	4500	12.2196	21.6352
1.8487	2.8395	4600	12.1848	21.6065
1.8157	2.9012	4700	12.2117	21.4347
1.8413	2.9630	4800	12.2470	21.3488
1.8274	3.0247	4900	12.2649	21.4920
1.8070	3.0864	5000	12.2594	21.0624
1.8142	3.1481	5100	12.2709	21.3488
1.8106	3.2099	5200	12.3053	20.9908
1.8164	3.2716	5300	12.2847	20.8047
1.8389	3.3333	5400	12.3290	20.9622
1.7915	3.3951	5500	12.2934	20.7045
1.8086	3.4568	5600	12.2837	20.5183
1.8164	3.5185	5700	12.3072	20.5183
1.8111	3.5802	5800	12.2692	20.2320
1.7957	3.6420	5900	12.2973	20.2033
1.8279	3.7037	6000	12.3178	19.9026
1.7774	3.7654	6100	12.3119	19.7308
1.8099	3.8272	6200	12.3148	19.5876
1.7823	3.8889	6300	12.3140	19.6163
1.8097	3.9506	6400	12.2987	19.5876
1.8135	4.0123	6500	12.3467	19.5017
1.7750	4.0741	6600	12.3299	19.6735
1.7952	4.1358	6700	12.3323	19.6019
1.7969	4.1975	6800	12.3599	19.6449
1.7807	4.2593	6900	12.3475	19.3872
1.7757	4.3210	7000	12.3626	19.3156
1.7890	4.3827	7100	12.3674	19.0149
1.7835	4.4444	7200	12.3707	19.1724
1.7669	4.5062	7300	12.3870	18.9576
1.7814	4.5679	7400	12.4070	19.0292
1.7606	4.6296	7500	12.4070	18.8431
1.7664	4.6914	7600	12.3646	18.6569
1.7843	4.7531	7700	12.3676	18.8001
1.7818	4.8148	7800	12.3788	18.7285
1.7587	4.8765	7900	12.4390	18.8574
1.7304	4.9383	8000	12.4128	18.6712
1.7494	5.0	8100	12.3993	18.8288
1.7382	5.0617	8200	12.3960	18.6999
1.7570	5.1235	8300	12.4306	18.2417
1.7582	5.1852	8400	12.4159	18.2703
1.7440	5.2469	8500	12.4073	18.1987
1.7320	5.3086	8600	12.4036	17.9553
1.7329	5.3704	8700	12.4001	18.0269
1.7838	5.4321	8800	12.4316	18.3419
1.7656	5.4938	8900	12.4561	17.8121
1.7398	5.5556	9000	12.4558	17.7692
1.7388	5.6173	9100	12.4364	17.8694
1.7277	5.6790	9200	12.4390	17.7978
1.7251	5.7407	9300	12.4184	17.4828
1.7138	5.8025	9400	12.4593	17.4971
1.7318	5.8642	9500	12.4649	17.7692
1.7525	5.9259	9600	12.4540	17.6833
1.7357	5.9877	9700	12.4707	17.6260
1.7266	6.0494	9800	12.4238	17.4542
1.7412	6.1111	9900	12.4467	17.5258
1.7125	6.1728	10000	12.4267	17.4828
1.7362	6.2346	10100	12.4504	17.2394
1.7269	6.2963	10200	12.4637	17.1678
1.7021	6.3580	10300	12.4308	17.3969
1.7362	6.4198	10400	12.4621	17.2251
1.7280	6.4815	10500	12.4484	17.1821
1.7200	6.5432	10600	12.4573	17.0103
1.7058	6.6049	10700	12.4859	17.0389
1.7390	6.6667	10800	12.4640	17.0389
1.7201	6.7284	10900	12.5004	16.8528
1.7329	6.7901	11000	12.5149	16.6380
1.7209	6.8519	11100	12.4906	16.9244
1.7088	6.9136	11200	12.4817	16.9530
1.7152	6.9753	11300	12.5157	16.7812
1.7393	7.0370	11400	12.5130	16.6810
1.6985	7.0988	11500	12.4801	16.7239
1.7192	7.1605	11600	12.4938	16.5951
1.7145	7.2222	11700	12.5002	16.7239
1.7001	7.2840	11800	12.5122	16.7383
1.7200	7.3457	11900	12.4965	16.4233
1.7083	7.4074	12000	12.4749	16.6523
1.7161	7.4691	12100	12.5027	16.5092
1.7002	7.5309	12200	12.4944	16.5808
1.7136	7.5926	12300	12.5131	16.6810
1.7120	7.6543	12400	12.5127	16.5235
1.7002	7.7160	12500	12.4984	16.4805
1.7161	7.7778	12600	12.5244	16.1512
1.7042	7.8395	12700	12.4983	16.4233
1.6993	7.9012	12800	12.5019	16.2514
1.6861	7.9630	12900	12.5004	16.3230
1.6892	8.0247	13000	12.5112	16.3087
1.7014	8.0864	13100	12.5143	16.2514
1.7139	8.1481	13200	12.5307	16.0939
1.6975	8.2099	13300	12.5278	16.3946
1.6978	8.2716	13400	12.5044	16.2228
1.7316	8.3333	13500	12.4955	16.0080
1.6921	8.3951	13600	12.4934	16.1082
1.7163	8.4568	13700	12.5006	16.1798
1.6976	8.5185	13800	12.5102	15.9794
1.6966	8.5802	13900	12.5120	16.2514
1.7045	8.6420	14000	12.5177	16.0796
1.6783	8.7037	14100	12.5012	16.2228
1.6831	8.7654	14200	12.5125	15.9794
1.6936	8.8272	14300	12.5232	16.1082
1.6874	8.8889	14400	12.5295	16.2085
1.6933	8.9506	14500	12.5139	16.2228
1.7119	9.0123	14600	12.5230	16.0223
1.6975	9.0741	14700	12.5362	16.1512
1.6950	9.1358	14800	12.5241	16.0653
1.7160	9.1975	14900	12.5257	16.0939
1.6975	9.2593	15000	12.5181	16.0510

Framework versions

• Transformers 5.12.1
• Pytorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2