Hugging Face's logo

GetmanY1
/
wav2vec2-base-fi-lp-cont-pt-100h

Automatic Speech Recognition
Transformers
Safetensors
Finnish
wav2vec2
finnish
Eval Results
Inference Endpoints
Model card Files Community
wav2vec2-base-fi-lp-cont-pt-100h
File size: 6,494 Bytes 
c8978ea
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
702e095
c8978ea
44cd9f7
c8978ea
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ba44a84
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c8978ea
 
 
 
 
44cd9f7
 
 
 
 
 
 
 
c8978ea
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
 
---
license: apache-2.0
language: fi
tags:
- automatic-speech-recognition
- fi
- finnish
model-index:
  - name: wav2vec2-base-fi-lp-cont-pt-100h
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: Lahjoita puhetta (Donate Speech)
          type: lahjoita-puhetta
          args: fi
        metrics:
          - name: Dev WER
            type: wer
            value: 22.49
          - name: Dev CER
            type: cer
            value: 5.84
          - name: Test WER
            type: wer
            value: 24.95
          - name: Test CER
            type: cer
            value: 7.09
---
# Colloquial Finnish Wav2vec2-Base ASR

[GetmanY1/wav2vec2-base-fi-lp-cont-pt](https://huggingface.co/GetmanY1/wav2vec2-base-fi-lp-cont-pt) fine-tuned on 100 hours of [Lahjoita puhetta (Donate Speech)](https://link.springer.com/article/10.1007/s10579-022-09606-3) on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.

## Model description

The Finnish Wav2Vec2 Base has the same architecture and uses the same training objective as the English and multilingual one described in [Paper](https://arxiv.org/abs/2006.11477).

You can read more about the pre-trained model from [this paper](https://www.isca-archive.org/interspeech_2024/getman24_interspeech.html). The training scripts are available on [GitHub](https://github.com/aalto-speech/colloquial-Finnish-wav2vec2)

## Intended uses & limitations

You can use this model for Finnish ASR (speech-to-text). 

### How to use

To transcribe audio files the model can be used as a standalone acoustic model as follows:

```
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
from datasets import load_dataset
import torch

# load model and processor
processor = Wav2Vec2Processor.from_pretrained("GetmanY1/wav2vec2-base-fi-lp-cont-pt-100h")
model = Wav2Vec2ForCTC.from_pretrained("GetmanY1/wav2vec2-base-fi-lp-cont-pt-100h")

# load dummy dataset and read soundfiles
ds = load_dataset("mozilla-foundation/common_voice_16_1", "fi", split='test')

# tokenize
input_values = processor(ds[0]["audio"]["array"], return_tensors="pt", padding="longest").input_values  # Batch size 1

# retrieve logits
logits = model(input_values).logits

# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)
```

### Limitations and bias

This model was fine-tuned with audio samples whose maximum length was 50 seconds so this model most likely works the best for short audios of similar length. However, you can try this model with a lot longer audios too and see how it works. If you encounter out of memory errors with very long audio files you can use the audio chunking method introduced in [this blog post](https://huggingface.co/blog/asr-chunking).

The model was fine-tuned on the data from the [Lahjoita puhetta (Donate Speech) corpus](https://link.springer.com/article/10.1007/s10579-022-09606-3) so this model might have biases towards colloquial Finnish.

## Evaluation results

Evaluation results in terms of WER (word error rate) and CER (character error rate) on the Lahjoita puhetta dev and test sets:

|**System**                                                                                                                   | **Labeled training data, h** | **dev WER [%]** | **dev CER [%]**  | **test WER [%]**  | **test CER [%]**  |
|-----------------------------------------------------------------------------------------------------------------------------|------------------------------|-------------|--------------|---------------|---------------|
|**Base Models**
|[GetmanY1/wav2vec2-base-fi-voxpopuli-v2-100h](https://huggingface.co/GetmanY1/wav2vec2-base-fi-voxpopuli-v2-100h)            |100 |29.35|7.94|31.90|9.05|
|[GetmanY1/wav2vec2-base-fi-voxpopuli-v2-1500h](https://huggingface.co/GetmanY1/wav2vec2-base-fi-voxpopuli-v2-100h)           |1500|22.18|5.96|24.43|6.97|
|[GetmanY1/wav2vec2-base-fi-lp-from-scratch-100h](https://huggingface.co/GetmanY1/wav2vec2-base-fi-lp-from-scratch-100h)      |100 |26.40|6.86|28.92|8.09|
|[GetmanY1/wav2vec2-base-fi-lp-from-scratch-1500h](https://huggingface.co/GetmanY1/wav2vec2-base-fi-lp-from-scratch-1500h)    |1500|21.61|5.59|24.35|6.87|
|[GetmanY1/wav2vec2-base-fi-lp-cont-pt-100h](https://huggingface.co/GetmanY1/wav2vec2-base-fi-lp-cont-pt-100h)                |100 |22.49|5.84|24.95|7.09|
|[GetmanY1/wav2vec2-base-fi-lp-cont-pt-1500h](https://huggingface.co/GetmanY1/wav2vec2-base-fi-lp-cont-pt-1500h)              |1500|**17.38**|**4.61**|**19.65**|**5.69**|
|**Large Models**
|[GetmanY1/wav2vec2-large-uralic-voxpopuli-v2-100h](https://huggingface.co/GetmanY1/wav2vec2-large-uralic-voxpopuli-v2-100h)  |100 |21.02|5.70|22.98|6.90|
|[GetmanY1/wav2vec2-large-uralic-voxpopuli-v2-1500h](https://huggingface.co/GetmanY1/wav2vec2-large-uralic-voxpopuli-v2-1500h)|1500|19.14|5.05|20.49|5.93|
|[GetmanY1/wav2vec2-large-fi-lp-from-scratch-100h](https://huggingface.co/GetmanY1/wav2vec2-large-fi-lp-from-scratch-100h)    |100 |21.66|5.61|23.85|6.76|
|[GetmanY1/wav2vec2-large-fi-lp-from-scratch-1500h](https://huggingface.co/GetmanY1/wav2vec2-large-fi-lp-from-scratch-1500h)  |1500|17.54|4.59|19.26|5.58|
|[GetmanY1/wav2vec2-large-fi-lp-cont-pt-100h](https://huggingface.co/GetmanY1/wav2vec2-large-fi-lp-cont-pt-100h)              |100 |20.20|5.40|22.81|6.64|
|[GetmanY1/wav2vec2-large-fi-lp-cont-pt-1500h](https://huggingface.co/GetmanY1/wav2vec2-large-fi-lp-cont-pt-1500h)            |1500|**16.24**|**4.34**|**18.04**|**5.29**|

## Citation

If you use our models or scripts, please cite our article as:

```bibtex
@inproceedings{getman24_interspeech,
  title     = {What happens in continued pre-training? Analysis of self-supervised speech
 models with continued pre-training for colloquial Finnish ASR},
  author    = {Yaroslav Getman and Tamas Grosz and Mikko Kurimo},
  year      = {2024},
  booktitle = {Interspeech 2024},
  pages     = {5043--5047},
  doi       = {10.21437/Interspeech.2024-476},
}
```

## Team Members

- Yaroslav Getman, [Hugging Face profile](https://huggingface.co/GetmanY1), [LinkedIn profile](https://www.linkedin.com/in/yaroslav-getman/)
- Tamas Grosz, [Hugging Face profile](https://huggingface.co/Grosy), [LinkedIn profile](https://www.linkedin.com/in/tam%C3%A1s-gr%C3%B3sz-950a049a/)

Feel free to contact us for more details 🤗