mpasila commited on
Commit
7a9c9d5
1 Parent(s): d8d5c72

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +103 -0
README.md ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - fi
4
+ license: apache-2.0
5
+ tags:
6
+ - whisper-event
7
+ - finnish
8
+ - speech-recognition
9
+ datasets:
10
+ - mozilla-foundation/common_voice_11_0
11
+ - google/fleurs
12
+ metrics:
13
+ - wer
14
+ - cer
15
+ model-index:
16
+ - name: Whisper Large V3 Finnish
17
+ results:
18
+ - task:
19
+ name: Automatic Speech Recognition
20
+ type: automatic-speech-recognition
21
+ dataset:
22
+ name: Common Voice 11.0
23
+ type: mozilla-foundation/common_voice_11_0
24
+ config: fi
25
+ split: test
26
+ args: fi
27
+ metrics:
28
+ - name: Wer
29
+ type: wer
30
+ value: 8.23
31
+ - name: Cer
32
+ type: cer
33
+ value: 1.43
34
+ - task:
35
+ name: Automatic Speech Recognition
36
+ type: automatic-speech-recognition
37
+ dataset:
38
+ name: FLEURS
39
+ type: google/fleurs
40
+ config: fi_fi
41
+ split: test
42
+ args: fi_fi
43
+ metrics:
44
+ - name: Wer
45
+ type: wer
46
+ value: 8.21
47
+ - name: Cer
48
+ type: cer
49
+ value: 3.23
50
+ library_name: transformers
51
+ pipeline_tag: automatic-speech-recognition
52
+ ---
53
+ This is a conversion of [Finnish-NLP/whisper-large-finnish-v3](https://huggingface.co/Finnish-NLP/whisper-large-finnish-v3) into faster-whisper format.
54
+
55
+ <h3>This is our improved Whisper v3 model that is now finetuned from OpenAI Whisper Large V3 </h3>
56
+ <p>We improve from our previously finetuned Whisper V2 model in the following manner<a>https://huggingface.co/Finnish-NLP/whisper-large-v2-finnish</a> </p>
57
+ <p>CV11 (Common Voice 11 test set) WER (Word error rate) 10.42 --> 8.23</p>
58
+ <p>Fleurs (A speech recognition test set by Google) WER (Word error rate) 10.20 --> 8.21</p>
59
+ <p>Model was trained on Nvidia RTX4080 for 32k steps with batch size 8, gradient accumulation 2</p>
60
+
61
+ <br>
62
+
63
+ <h3> Original OpenAI Whisper Large V3</h3>
64
+ - CV11
65
+ - WER: 14.81
66
+ - WER NORMALIZED: 10.82
67
+ - CER: 2.7
68
+ - CER NORMALIZED: 2.07
69
+
70
+ - Fleurs
71
+ - WER: 12.04
72
+ - WER NORMALIZED: 9.63
73
+ - CER: 2.48
74
+ - CER NORMALIZED: 3.64
75
+
76
+
77
+ <h3> After Finetuning with Finnish data our V3 got these scores on the test set:</h3>
78
+
79
+ - @14000 finetuning steps
80
+ - CV11
81
+ - WER: 11.36
82
+ - WER NORMALIZED: 8.31
83
+ - CER: 1.93
84
+ - CER NORMALIZED: 1.48
85
+
86
+ - Fleurs
87
+ - WER: 10.2
88
+ - WER NORMALIZED: 8.56
89
+ - CER: 2.26
90
+ - CER NORMALIZED: 3.54
91
+
92
+ - @32000 finetuning steps
93
+ - CV11
94
+ - WER: 11.47
95
+ - WER NORMALIZED: 8.23
96
+ - CER: 1.91
97
+ - CER NORMALIZED: 1.43
98
+
99
+ - Fleurs
100
+ - WER: 10.1
101
+ - WER NORMALIZED: 8.21
102
+ - CER: 2.2
103
+ - CER NORMALIZED: 3.23