GetmanY1 commited on
Commit
05d23e3
1 Parent(s): bd54afa

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +101 -0
README.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language: fi
4
+ tags:
5
+ - automatic-speech-recognition
6
+ - fi
7
+ - finnish
8
+ model-index:
9
+ - name: wav2vec2-large-fi-lp-cont-pt-1500h
10
+ results:
11
+ - task:
12
+ name: Automatic Speech Recognition
13
+ type: automatic-speech-recognition
14
+ dataset:
15
+ name: Lahjoita puhetta (Donate Speech)
16
+ type: lahjoita-puhetta
17
+ args: fi
18
+ metrics:
19
+ - name: Dev WER
20
+ type: wer
21
+ value: 16.24
22
+ - name: Dev CER
23
+ type: cer
24
+ value: 4.34
25
+ - name: Test WER
26
+ type: wer
27
+ value: 18.04
28
+ - name: Test CER
29
+ type: cer
30
+ value: 5.29
31
+ ---
32
+ # Colloquial Finnish Wav2vec2-Large ASR
33
+
34
+ [GetmanY1/wav2vec2-large-fi-lp-cont-pt](https://huggingface.co/GetmanY1/wav2vec2-large-fi-lp-cont-pt) fine-tuned on 1500 hours of [Lahjoita puhetta (Donate Speech)](https://link.springer.com/article/10.1007/s10579-022-09606-3) on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.
35
+
36
+ ## Model description
37
+
38
+ The Finnish Wav2Vec2 Large has the same architecture and uses the same training objective as the English and multilingual one described in [Paper](https://arxiv.org/abs/2006.11477). It is pre-trained on 2600 hours of unlabeled colloquial Finnish speech from [Lahjoita puhetta (Donate Speech)](https://link.springer.com/article/10.1007/s10579-022-09606-3).
39
+
40
+ You can read more about the pre-trained model from [this paper](TODO). The training scripts are available on [GitHub](https://github.com/aalto-speech/colloquial-Finnish-wav2vec2)
41
+
42
+ ## Intended uses & limitations
43
+
44
+ You can use this model for Finnish ASR (speech-to-text).
45
+
46
+ ### How to use
47
+
48
+ To transcribe audio files the model can be used as a standalone acoustic model as follows:
49
+
50
+ ```
51
+ from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
52
+ from datasets import load_dataset
53
+ import torch
54
+
55
+ # load model and processor
56
+ processor = Wav2Vec2Processor.from_pretrained("GetmanY1/wav2vec2-large-fi-lp-cont-pt-1500h")
57
+ model = Wav2Vec2ForCTC.from_pretrained("GetmanY1/wav2vec2-large-fi-lp-cont-pt-1500h")
58
+
59
+ # load dummy dataset and read soundfiles
60
+ ds = load_dataset("mozilla-foundation/common_voice_16_1", "fi", split='test')
61
+
62
+ # tokenize
63
+ input_values = processor(ds[0]["audio"]["array"], return_tensors="pt", padding="longest").input_values # Batch size 1
64
+
65
+ # retrieve logits
66
+ logits = model(input_values).logits
67
+
68
+ # take argmax and decode
69
+ predicted_ids = torch.argmax(logits, dim=-1)
70
+ transcription = processor.batch_decode(predicted_ids)
71
+ ```
72
+
73
+ ### Limitations and bias
74
+
75
+ This model was fine-tuned with audio samples whose maximum length was 50 seconds so this model most likely works the best for short audios of similar length. However, you can try this model with a lot longer audios too and see how it works. If you encounter out of memory errors with very long audio files you can use the audio chunking method introduced in [this blog post](https://huggingface.co/blog/asr-chunking).
76
+
77
+ The model was fine-tuned on the data from the [Lahjoita puhetta (Donate Speech) corpus](https://link.springer.com/article/10.1007/s10579-022-09606-3) so this model might have biases towards colloquial Finnish.
78
+
79
+ ## Citation
80
+
81
+ If you use our models or scripts, please cite our article as:
82
+
83
+ ```bibtex
84
+ @inproceedings{getman24a_interspeech,
85
+ author={Yaroslav Getman and Tamas Grosz and Mikko Kurimo},
86
+ title={{What happens in continued pre-training? Analysis of self-supervised speech
87
+ models with continued pre-training for colloquial Finnish ASR}},
88
+ year=2024,
89
+ booktitle={Proc. INTERSPEECH 2024},
90
+ pages={XX--XX},
91
+ doi={XXXX},
92
+ issn={XXXX-XXXX}
93
+ }
94
+ ```
95
+
96
+ ## Team Members
97
+
98
+ - Yaroslav Getman, [Hugging Face profile](https://huggingface.co/GetmanY1), [LinkedIn profile](https://www.linkedin.com/in/yaroslav-getman/)
99
+ - Tamas Grosz, [Hugging Face profile](https://huggingface.co/Grosy), [LinkedIn profile](https://www.linkedin.com/in/tam%C3%A1s-gr%C3%B3sz-950a049a/)
100
+
101
+ Feel free to contact us for more details 🤗