Hubert-common_voice-ja-demo-roma-debug-40epochs-cosine

This model is a fine-tuned version of rinna/japanese-hubert-base on the MOZILLA-FOUNDATION/COMMON_VOICE_13_0 - JA dataset. It achieves the following results on the evaluation set:

• Loss: 0.5318
• Wer: 0.9990
• Cer: 0.1993

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500
• num_epochs: 40.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
No log	0.2660	100	16.8172	2.9494	3.4779
No log	0.5319	200	16.5502	2.7606	2.9561
No log	0.7979	300	15.8340	1.9101	1.6839
No log	1.0638	400	13.2919	1.0	0.9276
12.6588	1.3298	500	7.8792	1.0	0.9276
12.6588	1.5957	600	6.1542	1.0	0.9276
12.6588	1.8617	700	5.7757	1.0	0.9276
12.6588	2.1277	800	5.6188	1.0	0.9276
12.6588	2.3936	900	5.4753	1.0	0.9276
5.2353	2.6596	1000	5.3239	1.0	0.9276
5.2353	2.9255	1100	5.1676	1.0	0.9276
5.2353	3.1915	1200	5.0084	1.0	0.9276
5.2353	3.4574	1300	4.8402	1.0	0.9276
5.2353	3.7234	1400	4.6702	1.0	0.9276
4.4502	3.9894	1500	4.4957	1.0	0.9276
4.4502	4.2553	1600	4.3219	1.0	0.9276
4.4502	4.5213	1700	4.1502	1.0	0.9276
4.4502	4.7872	1800	3.9856	1.0	0.9276
4.4502	5.0532	1900	3.8343	1.0	0.9276
3.7863	5.3191	2000	3.6907	1.0	0.9276
3.7863	5.5851	2100	3.5544	1.0	0.9276
3.7863	5.8511	2200	3.4332	1.0	0.9276
3.7863	6.1170	2300	3.3063	1.0	0.9276
3.7863	6.3830	2400	3.2075	1.0	0.9276
3.2473	6.6489	2500	3.1272	1.0	0.9276
3.2473	6.9149	2600	3.0657	1.0	0.9276
3.2473	7.1809	2700	3.0164	1.0	0.9276
3.2473	7.4468	2800	2.9748	1.0	0.9276
3.2473	7.7128	2900	2.9447	1.0	0.9276
2.9649	7.9787	3000	2.9188	1.0	0.9276
2.9649	8.2447	3100	2.9006	1.0	0.9276
2.9649	8.5106	3200	2.8877	1.0	0.9276
2.9649	8.7766	3300	2.8712	1.0	0.9276
2.9649	9.0426	3400	2.8529	1.0	0.9276
2.8673	9.3085	3500	2.8439	1.0	0.9276
2.8673	9.5745	3600	2.8313	1.0	0.9276
2.8673	9.8404	3700	2.8182	1.0	0.9276
2.8673	10.1064	3800	2.7311	1.0	0.9276
2.8673	10.3723	3900	2.4997	1.0	0.9276
2.6801	10.6383	4000	2.2398	1.0	0.8951
2.6801	10.9043	4100	1.9111	1.0	0.6154
2.6801	11.1702	4200	1.5447	1.0	0.4341
2.6801	11.4362	4300	1.3182	1.0	0.3959
2.6801	11.7021	4400	1.1702	0.9996	0.3706
1.5214	11.9681	4500	1.0558	0.9992	0.3214
1.5214	12.2340	4600	0.9717	0.9988	0.3024
1.5214	12.5	4700	0.8959	0.9982	0.2874
1.5214	12.7660	4800	0.8399	0.9978	0.2747
1.5214	13.0319	4900	0.7891	0.9974	0.2657
0.8719	13.2979	5000	0.7484	0.9980	0.2580
0.8719	13.5638	5100	0.7145	0.9976	0.2523
0.8719	13.8298	5200	0.6852	0.9976	0.2481
0.8719	14.0957	5300	0.6618	0.9980	0.2487
0.8719	14.3617	5400	0.6400	0.9986	0.2477
0.6568	14.6277	5500	0.6200	0.9988	0.2449
0.6568	14.8936	5600	0.6032	0.9988	0.2421
0.6568	15.1596	5700	0.5875	0.9984	0.2395
0.6568	15.4255	5800	0.5776	0.9990	0.2409
0.6568	15.6915	5900	0.5617	0.9994	0.2360
0.548	15.9574	6000	0.5485	0.9982	0.2347
0.548	16.2234	6100	0.5394	0.9988	0.2334
0.548	16.4894	6200	0.5322	0.9984	0.2317
0.548	16.7553	6300	0.5243	0.9970	0.2320
0.548	17.0213	6400	0.5121	0.9986	0.2272
0.4681	17.2872	6500	0.5070	0.9990	0.2266
0.4681	17.5532	6600	0.5014	0.9992	0.2263
0.4681	17.8191	6700	0.4943	0.9986	0.2242
0.4681	18.0851	6800	0.4930	0.9988	0.2228
0.4681	18.3511	6900	0.4969	0.9986	0.2245
0.4198	18.6170	7000	0.4883	0.9986	0.2225
0.4198	18.8830	7100	0.4805	0.9986	0.2215
0.4198	19.1489	7200	0.4777	0.9984	0.2208
0.4198	19.4149	7300	0.4718	0.9988	0.2209
0.4198	19.6809	7400	0.4721	0.9984	0.2199
0.3795	19.9468	7500	0.4675	0.9984	0.2205
0.3795	20.2128	7600	0.4692	0.9988	0.2162
0.3795	20.4787	7700	0.4732	0.9986	0.2173
0.3795	20.7447	7800	0.4654	0.9982	0.2173
0.3795	21.0106	7900	0.4557	0.9986	0.2158
0.3504	21.2766	8000	0.4562	0.9982	0.2144
0.3504	21.5426	8100	0.4679	0.9982	0.2144
0.3504	21.8085	8200	0.4584	0.9990	0.2169
0.3504	22.0745	8300	0.4561	0.9982	0.2134
0.3504	22.3404	8400	0.4595	0.9988	0.2143
0.3134	22.6064	8500	0.4544	0.9986	0.2155
0.3134	22.8723	8600	0.4544	0.9984	0.2134
0.3134	23.1383	8700	0.4552	0.9984	0.2129
0.3134	23.4043	8800	0.4524	0.9984	0.2121
0.3134	23.6702	8900	0.4554	0.9986	0.2113
0.3014	23.9362	9000	0.4617	0.9982	0.2103
0.3014	24.2021	9100	0.4606	0.9978	0.2130
0.3014	24.4681	9200	0.4561	0.9974	0.2105
0.3014	24.7340	9300	0.4566	0.9984	0.2089
0.3014	25.0	9400	0.4486	0.9990	0.2119
0.2791	25.2660	9500	0.4542	0.9990	0.2117
0.2791	25.5319	9600	0.4540	0.9986	0.2095
0.2791	25.7979	9700	0.4419	0.9984	0.2091
0.2791	26.0638	9800	0.4569	0.9982	0.2074
0.2791	26.3298	9900	0.4543	0.9984	0.2090
0.2564	26.5957	10000	0.4689	0.9982	0.2088
0.2564	26.8617	10100	0.4590	0.9984	0.2089
0.2564	27.1277	10200	0.4986	0.9986	0.2093
0.2564	27.3936	10300	0.4693	0.9990	0.2100
0.2564	27.6596	10400	0.5128	0.9982	0.2085
0.2449	27.9255	10500	0.4512	0.9984	0.2099
0.2449	28.1915	10600	0.4651	0.9994	0.2091
0.2449	28.4574	10700	0.4604	0.9984	0.2068
0.2449	28.7234	10800	0.4687	0.9990	0.2080
0.2449	28.9894	10900	0.4688	0.9994	0.2064
0.2258	29.2553	11000	0.4759	0.9994	0.2092
0.2258	29.5213	11100	0.4816	0.9988	0.2068
0.2258	29.7872	11200	0.4750	0.9988	0.2053
0.2258	30.0532	11300	0.4753	0.9986	0.2048
0.2258	30.3191	11400	0.4829	0.9992	0.2060
0.2124	30.5851	11500	0.4800	0.9986	0.2081
0.2124	30.8511	11600	0.5290	0.9990	0.2061
0.2124	31.1170	11700	0.5369	0.9988	0.2055
0.2124	31.3830	11800	0.5170	0.9978	0.2041
0.2124	31.6489	11900	0.5229	0.9990	0.2070
0.2007	31.9149	12000	0.5035	0.9986	0.2060
0.2007	32.1809	12100	0.5103	0.9974	0.2049
0.2007	32.4468	12200	0.4868	0.9972	0.2032
0.2007	32.7128	12300	0.4867	0.9996	0.2043
0.2007	32.9787	12400	0.5049	0.9982	0.2040
0.1867	33.2447	12500	0.5126	0.9984	0.2040
0.1867	33.5106	12600	0.5321	0.9992	0.2037
0.1867	33.7766	12700	0.5187	0.9978	0.2040
0.1867	34.0426	12800	0.5319	0.9990	0.2064
0.1867	34.3085	12900	0.5275	0.9980	0.2041
0.1749	34.5745	13000	0.5433	0.9982	0.2043
0.1749	34.8404	13100	0.5094	0.9984	0.2023
0.1749	35.1064	13200	0.5363	0.9990	0.2004
0.1749	35.3723	13300	0.5331	0.9994	0.2022
0.1749	35.6383	13400	0.5053	0.9990	0.2009
0.1604	35.9043	13500	0.5157	0.9990	0.2026
0.1604	36.1702	13600	0.5299	0.9990	0.2018
0.1604	36.4362	13700	0.5117	0.9996	0.2050
0.1604	36.7021	13800	0.5067	0.9994	0.2038
0.1604	36.9681	13900	0.4994	0.9996	0.2028
0.1412	37.2340	14000	0.5346	0.9984	0.2024
0.1412	37.5	14100	0.5350	0.9994	0.2015
0.1412	37.7660	14200	0.5237	0.9990	0.2010
0.1412	38.0319	14300	0.5305	0.9992	0.1993
0.1412	38.2979	14400	0.5309	0.9986	0.1973
0.1286	38.5638	14500	0.5270	0.9992	0.1992
0.1286	38.8298	14600	0.5363	0.9990	0.1999
0.1286	39.0957	14700	0.5347	0.9990	0.1999
0.1286	39.3617	14800	0.5319	0.9990	0.1999
0.1286	39.6277	14900	0.5322	0.9994	0.1995
0.1217	39.8936	15000	0.5322	0.9992	0.1992

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3