efwkjn/cohere-asr-ja

Cohere finetune for Japanese focused on general/anime domains.

From v0.1, tuned hyperparameters improving CER across all test sets. Adjusted loss resulting in slightly higher insertion rate on certain audios but I think the insertions are nicer.

• GSPO with CE
• 2^17 steps
• 64 bsz
• Each sequence sample 1 greedy/7 temp1,topk20
• Audio length 0-35s, avg ~25.3s
• Frozen encoder, full finetune decoder

Bigger but faster than whisper-large-v3-turbo from more subsampling. Likely final checkpoint, will focus on finishing whisper training.

See my most recent whisper finetune for more evals

	tedx					jsut-book
	cer	sr	ir	dr		cer	sr	ir	dr
cohere	22.0	5.0	6.0	11.1		29.8	9.9	1.9	18.0
v0.1	14.4	4.7	7.5	2.2		18.3	11.2	0.9	6.3
ja	12.9	4.8	6.0	2.1		17.5	10.9	0.8	5.8

cv/reazon removed due to contamination

	fleurs					jsut-basic					blue-archive					nekopara
	cer	sr	ir	dr		cer	sr	ir	dr		cer	sr	ir	dr		cer	sr	ir	dr
cohere_b1	4.9	2.7	0.4	1.8		8.4	5.7	1.8	0.8		8.1	4.0	1.2	2.9		30.4	13.5	3.7	13.2
cohere_b5	4.9	2.6	0.4	1.9		8.4	5.7	1.8	0.9		8.1	3.9	1.1	3.1		29.4	12.7	2.0	14.8
v0.1_b1	5.2	2.8	0.4	1.9		6.9	5.0	1.1	0.8		6.8	3.9	1.4	1.5		24.8	14.0	3.1	7.7
v0.1_b5	4.9	2.7	0.4	1.9		6.8	4.9	1.1	0.8		6.7	3.7	1.2	1.7		24.2	12.9	1.8	9.5
ja_b1	5.0	2.8	0.5	1.7		6.7	4.9	1.0	0.8		6.6	3.8	1.5	1.4		24.4	14.1	3.5	6.8
ja_b5	4.8	2.7	0.4	1.7		6.6	4.8	1.0	0.8		6.5	3.7	1.3	1.5		23.4	12.8	1.9	8.7

	air					clannad					himanatsu					kanon					sakuuta
	cer	sr	ir	dr		cer	sr	ir	dr		cer	sr	ir	dr		cer	sr	ir	dr		cer	sr	ir	dr
cohere_b1	18.2	9.4	1.5	7.3		17.5	7.7	2.0	7.8		28.1	11.1	5.3	11.7		16.8	8.1	2.2	6.5		16.1	7.7	3.3	5.1
cohere_b5	18.1	9.2	1.4	7.5		17.6	7.5	1.8	8.2		25.6	10.1	2.4	13.1		16.7	7.9	2.0	6.8		15.5	7.3	2.4	5.8
v0.1_b1	16.0	9.3	1.6	5.0		12.5	6.8	1.2	4.4		20.6	12.2	3.5	4.9		14.2	8.0	1.6	4.5		10.9	6.7	2.2	2.1
v0.1_b5	15.8	9.3	1.5	5.0		12.4	6.7	1.1	4.6		20.3	11.9	2.9	5.5		14.2	8.1	1.5	4.6		10.8	6.5	2.0	2.2
ja_b1	15.8	9.2	1.6	5.0		12.3	6.7	1.2	4.4		20.5	12.1	3.5	4.9		13.9	7.9	1.6	4.4		10.8	6.6	2.2	2.1
ja_b5	15.4	9.0	1.3	5.2		12.1	6.5	0.9	4.7		19.7	11.3	2.3	6.0		13.9	7.9	1.3	4.7		10.6	6.3	1.9	2.3