hubert_model

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2139
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 64
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 125
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
55.5107	0.11	100	93.6947	1.0
29.8329	0.22	200	53.0718	1.0
22.4958	0.32	300	42.6961	1.0
19.1734	0.43	400	34.1686	1.0
15.9615	0.54	500	27.1054	1.0
13.1077	0.65	600	21.2901	1.0
11.0162	0.76	700	16.6558	1.0
9.3359	0.87	800	13.1283	1.0
8.2754	0.97	900	10.6005	1.0
7.1321	1.08	1000	8.7120	1.0
6.2621	1.19	1100	7.4866	1.0
5.8109	1.3	1200	6.6416	1.0
5.386	1.41	1300	6.1307	1.0
5.1782	1.51	1400	5.8103	1.0
4.9481	1.62	1500	5.6119	1.0
4.8722	1.73	1600	5.4872	1.0
4.7617	1.84	1700	5.3270	1.0
4.717	1.95	1800	5.2877	1.0
4.6256	2.06	1900	5.6727	1.0
4.6255	2.16	2000	5.4983	1.0
4.5977	2.27	2100	5.2167	1.0
4.5797	2.38	2200	4.9743	1.0
4.5616	2.49	2300	4.8446	1.0
4.5476	2.6	2400	4.7885	1.0
4.5516	2.71	2500	4.7597	1.0
4.5343	2.81	2600	4.7309	1.0
4.586	2.92	2700	4.7173	1.0
4.5813	3.03	2800	4.6650	1.0
4.4794	3.14	2900	4.5851	1.0
4.4735	3.25	3000	4.5310	1.0
4.4748	3.35	3100	4.5285	1.0
4.4439	3.46	3200	4.4971	1.0
4.4255	3.57	3300	4.5072	1.0
4.4232	3.68	3400	4.4936	1.0
4.4066	3.79	3500	4.5279	1.0
4.4095	3.9	3600	4.4653	1.0
4.3148	4.0	3700	4.4542	1.0
4.2788	4.11	3800	4.3820	1.0
4.3291	4.22	3900	4.3234	1.0
4.2173	4.33	4000	4.3068	1.0
4.1921	4.44	4100	4.2719	1.0
4.1868	4.55	4200	4.2765	1.0
4.1734	4.65	4300	4.2349	1.0
4.1868	4.76	4400	4.2002	1.0
4.169	4.87	4500	4.1915	1.0
4.131	4.98	4600	4.1673	1.0
4.1952	5.09	4700	4.1657	1.0
4.1067	5.19	4800	4.1650	1.0
4.1026	5.3	4900	4.1394	1.0
4.0864	5.41	5000	4.1334	1.0
4.0745	5.52	5100	4.1138	1.0
4.0653	5.63	5200	4.1029	1.0
4.0484	5.74	5300	4.0870	1.0
4.0474	5.84	5400	4.0693	1.0
4.0299	5.95	5500	4.0489	1.0
4.0862	6.06	5600	4.0414	1.0
3.9986	6.17	5700	4.0316	1.0
4.0042	6.28	5800	4.0100	1.0
3.9912	6.39	5900	3.9861	1.0
3.9625	6.49	6000	3.9676	1.0
3.96	6.6	6100	3.9469	1.0
3.9443	6.71	6200	3.9514	1.0
3.9215	6.82	6300	3.9108	1.0
3.9176	6.93	6400	3.8880	1.0
3.9986	7.03	6500	3.8798	1.0
3.8908	7.14	6600	3.8610	1.0
3.8715	7.25	6700	3.8430	1.0
3.8751	7.36	6800	3.8144	1.0
3.8643	7.47	6900	3.7939	1.0
3.8325	7.58	7000	3.7716	1.0
3.8269	7.68	7100	3.7620	1.0
3.82	7.79	7200	3.7440	1.0
3.8037	7.9	7300	3.7141	1.0
3.7488	8.01	7400	3.6912	1.0
3.7706	8.12	7500	3.6651	1.0
3.7454	8.22	7600	3.6520	1.0
3.748	8.33	7700	3.6190	1.0
3.7375	8.44	7800	3.6024	1.0
3.7045	8.55	7900	3.5830	1.0
3.6915	8.66	8000	3.5455	1.0
3.6647	8.77	8100	3.5333	1.0
3.645	8.87	8200	3.5053	1.0
3.6229	8.98	8300	3.4728	1.0
3.6574	9.09	8400	3.4310	1.0
3.6235	9.2	8500	3.4228	1.0
3.5773	9.31	8600	3.3695	1.0
3.5876	9.42	8700	3.3636	1.0
3.5547	9.52	8800	3.3299	1.0
3.5691	9.63	8900	3.3324	1.0
3.5284	9.74	9000	3.2827	1.0
3.4919	9.85	9100	3.2855	1.0
3.4769	9.96	9200	3.2446	1.0
3.4516	10.06	9300	3.2290	1.0
3.4402	10.17	9400	3.2170	1.0
3.3962	10.28	9500	3.1936	1.0
3.4377	10.39	9600	3.1687	1.0
3.3816	10.5	9700	3.1436	1.0
3.3902	10.61	9800	3.1505	1.0
3.4016	10.71	9900	3.1450	1.0
3.3716	10.82	10000	3.1074	1.0
3.3278	10.93	10100	3.0856	1.0
3.3598	11.04	10200	3.0711	1.0
3.3327	11.15	10300	3.0770	1.0
3.2911	11.26	10400	3.0325	1.0
3.2904	11.36	10500	2.9986	1.0
3.2709	11.47	10600	2.9960	1.0
3.2437	11.58	10700	2.9695	1.0
3.2532	11.69	10800	2.9565	1.0
3.2359	11.8	10900	2.9660	1.0
3.227	11.9	11000	2.9494	1.0
3.2292	12.01	11100	2.9384	1.0
3.197	12.12	11200	2.9342	1.0
3.183	12.23	11300	2.9108	1.0
3.1583	12.34	11400	2.8785	1.0
3.1501	12.45	11500	2.8748	1.0
3.1695	12.55	11600	2.8649	1.0
3.1341	12.66	11700	2.8779	1.0
3.141	12.77	11800	2.8420	1.0
3.113	12.88	11900	2.8088	1.0
3.1242	12.99	12000	2.7891	1.0
3.1234	13.1	12100	2.7859	1.0
3.1063	13.2	12200	2.7808	1.0
3.0785	13.31	12300	2.7735	1.0
3.0778	13.42	12400	2.7591	1.0
3.0559	13.53	12500	2.7519	1.0
3.046	13.64	12600	2.7228	1.0
3.0558	13.74	12700	2.7294	1.0
3.0489	13.85	12800	2.7090	1.0
3.0287	13.96	12900	2.7024	1.0
2.9927	14.07	13000	2.6963	1.0
2.9912	14.18	13100	2.6688	1.0
2.9816	14.29	13200	2.6834	1.0
2.966	14.39	13300	2.6762	1.0
2.9625	14.5	13400	2.6657	1.0
2.9827	14.61	13500	2.6598	1.0
2.9538	14.72	13600	2.6407	1.0
2.9524	14.83	13700	2.6399	1.0
2.9379	14.93	13800	2.6179	1.0
3.0388	15.04	13900	2.6130	1.0
2.9352	15.15	14000	2.6224	1.0
2.9172	15.26	14100	2.5905	1.0
2.9082	15.37	14200	2.5991	1.0
2.9566	15.48	14300	2.6069	1.0
2.9068	15.58	14400	2.5780	1.0
2.8904	15.69	14500	2.5782	1.0
2.8644	15.8	14600	2.5583	1.0
2.8932	15.91	14700	2.5593	1.0
2.8795	16.02	14800	2.5365	1.0
2.9069	16.13	14900	2.5330	1.0
2.9361	16.23	15000	2.5361	1.0
2.8348	16.34	15100	2.5445	1.0
2.851	16.45	15200	2.5289	1.0
2.864	16.56	15300	2.5193	1.0
2.8703	16.67	15400	2.5170	1.0
2.8326	16.77	15500	2.5213	1.0
2.8865	16.88	15600	2.5121	1.0
2.8495	16.99	15700	2.4892	1.0
2.8127	17.1	15800	2.4909	1.0
2.9142	17.21	15900	2.4761	1.0
2.7825	17.32	16000	2.4887	1.0
2.8134	17.42	16100	2.4658	1.0
2.826	17.53	16200	2.4658	1.0
2.812	17.64	16300	2.4666	1.0
2.7825	17.75	16400	2.4539	1.0
2.7964	17.86	16500	2.4550	1.0
2.8023	17.96	16600	2.4428	1.0
2.7691	18.07	16700	2.4448	1.0
2.7506	18.18	16800	2.4347	1.0
2.7784	18.29	16900	2.4214	1.0
2.755	18.4	17000	2.4309	1.0
2.7511	18.51	17100	2.4283	1.0
2.7425	18.61	17200	2.4294	1.0
2.7774	18.72	17300	2.4062	1.0
2.749	18.83	17400	2.4113	1.0
2.7407	18.94	17500	2.3999	1.0
2.7492	19.05	17600	2.4046	1.0
2.7538	19.16	17700	2.3945	1.0
2.7207	19.26	17800	2.3851	1.0
2.7176	19.37	17900	2.3954	1.0
2.7333	19.48	18000	2.3855	1.0
2.7192	19.59	18100	2.3802	1.0
2.7252	19.7	18200	2.3535	1.0
2.7002	19.8	18300	2.3808	1.0
2.6591	19.91	18400	2.3590	1.0
2.7684	20.02	18500	2.3627	1.0
2.6802	20.13	18600	2.3468	1.0
2.6649	20.24	18700	2.3405	1.0
2.6886	20.35	18800	2.3358	1.0
2.7023	20.45	18900	2.3514	1.0
2.6993	20.56	19000	2.3433	1.0
2.691	20.67	19100	2.3498	1.0
2.6666	20.78	19200	2.3457	1.0
2.6829	20.89	19300	2.3451	1.0
2.7203	21.0	19400	2.3287	1.0
2.6738	21.1	19500	2.3205	1.0
2.6781	21.21	19600	2.3264	1.0
2.7018	21.32	19700	2.3217	1.0
2.6642	21.43	19800	2.3114	1.0
2.6662	21.54	19900	2.3188	1.0
2.6636	21.64	20000	2.3180	1.0
2.6553	21.75	20100	2.3095	1.0
2.6369	21.86	20200	2.3066	1.0
2.6355	21.97	20300	2.3048	1.0
2.6317	22.08	20400	2.3080	1.0
2.6631	22.19	20500	2.2931	1.0
2.6469	22.29	20600	2.2910	1.0
2.6401	22.4	20700	2.2857	1.0
2.6434	22.51	20800	2.2951	1.0
2.635	22.62	20900	2.2924	1.0
2.637	22.73	21000	2.2831	1.0
2.6249	22.84	21100	2.2897	1.0
2.6293	22.94	21200	2.2790	1.0
2.6482	23.05	21300	2.2821	1.0
2.6204	23.16	21400	2.2709	1.0
2.6337	23.27	21500	2.2675	1.0
2.6339	23.38	21600	2.2658	1.0
2.6169	23.48	21700	2.2701	1.0
2.6038	23.59	21800	2.2774	1.0
2.6255	23.7	21900	2.2740	1.0
2.6029	23.81	22000	2.2777	1.0
2.6045	23.92	22100	2.2663	1.0
2.6367	24.03	22200	2.2627	1.0
2.6071	24.13	22300	2.2574	1.0
2.6057	24.24	22400	2.2477	1.0
2.6167	24.35	22500	2.2592	1.0
2.607	24.46	22600	2.2514	1.0
2.5864	24.57	22700	2.2514	1.0
2.6053	24.67	22800	2.2475	1.0
2.616	24.78	22900	2.2436	1.0
2.5876	24.89	23000	2.2511	1.0
2.5977	25.0	23100	2.2461	1.0
2.6238	25.11	23200	2.2404	1.0
2.566	25.22	23300	2.2471	1.0
2.5851	25.32	23400	2.2444	1.0
2.5916	25.43	23500	2.2402	1.0
2.6528	25.54	23600	2.2418	1.0
2.5831	25.65	23700	2.2314	1.0
2.5725	25.76	23800	2.2433	1.0
2.5842	25.87	23900	2.2260	1.0
2.604	25.97	24000	2.2392	1.0
2.5801	26.08	24100	2.2339	1.0
2.5798	26.19	24200	2.2354	1.0
2.5747	26.3	24300	2.2305	1.0
2.5879	26.41	24400	2.2272	1.0
2.5494	26.51	24500	2.2319	1.0
2.5789	26.62	24600	2.2228	1.0
2.573	26.73	24700	2.2305	1.0
2.5864	26.84	24800	2.2254	1.0
2.5658	26.95	24900	2.2154	1.0
2.5766	27.06	25000	2.2209	1.0
2.5468	27.16	25100	2.2197	1.0
2.5867	27.27	25200	2.2148	1.0
2.5573	27.38	25300	2.2282	1.0
2.5742	27.49	25400	2.2245	1.0
2.5537	27.6	25500	2.2233	1.0
2.5518	27.71	25600	2.2207	1.0
2.5823	27.81	25700	2.2125	1.0
2.5611	27.92	25800	2.2198	1.0
2.5933	28.03	25900	2.2153	1.0
2.5271	28.14	26000	2.2138	1.0
2.5768	28.25	26100	2.2167	1.0
2.5649	28.35	26200	2.2108	1.0
2.5522	28.46	26300	2.2150	1.0
2.5723	28.57	26400	2.2162	1.0
2.5799	28.68	26500	2.2145	1.0
2.5673	28.79	26600	2.2153	1.0
2.5584	28.9	26700	2.2171	1.0
2.5547	29.0	26800	2.2100	1.0
2.5643	29.11	26900	2.2104	1.0
2.6011	29.22	27000	2.2113	1.0
2.5506	29.33	27100	2.2171	1.0
2.5858	29.44	27200	2.2129	1.0
2.5437	29.55	27300	2.2138	1.0
2.5627	29.65	27400	2.2167	1.0
2.5552	29.76	27500	2.2144	1.0
2.5578	29.87	27600	2.2145	1.0
2.5628	29.98	27700	2.2139	1.0

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.0
• Datasets 2.7.1
• Tokenizers 0.13.2