HiTZ
/

Automatic Speech Recognition
NeMo
PyTorch
Basque
speech
audio
Transducer
Conformer
NeMo
Transformer
Eval Results
asierhv commited on
Commit
258f478
1 Parent(s): 6b741c7

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +176 -0
README.md ADDED
@@ -0,0 +1,176 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ language:
4
+ - eu
5
+ library_name: nemo
6
+ datasets:
7
+ - mozilla-foundation/common_voice_16_1
8
+ - gttsehu/basque_parliament_1
9
+ - openslr
10
+ metrics:
11
+ - wer
12
+ pipeline_tag: automatic-speech-recognition
13
+ tags:
14
+ - automatic-speech-recognition
15
+ - speech
16
+ - audio
17
+ - Transducer
18
+ - Conformer
19
+ - NeMo
20
+ - pytorch
21
+ - Transformer
22
+ model-index:
23
+ - name: stt_eu_conformer_transducer_large
24
+ results:
25
+ - task:
26
+ type: Automatic Speech Recognition
27
+ name: speech-recognition
28
+ dataset:
29
+ name: Mozilla Common Voice 16.1
30
+ type: mozilla-foundation/common_voice_16_1
31
+ config: eu
32
+ split: test
33
+ args:
34
+ language: eu
35
+ metrics:
36
+ - name: Test WER
37
+ type: wer
38
+ value: 2.79
39
+ - task:
40
+ type: Automatic Speech Recognition
41
+ name: speech-recognition
42
+ dataset:
43
+ name: Basque Parliament
44
+ type: gttsehu/basque_parliament_1
45
+ config: eu
46
+ split: test
47
+ args:
48
+ language: eu
49
+ metrics:
50
+ - name: Test WER
51
+ type: wer
52
+ value: 3.83
53
+ - task:
54
+ type: Automatic Speech Recognition
55
+ name: speech-recognition
56
+ dataset:
57
+ name: Basque Parliament
58
+ type: gttsehu/basque_parliament_1
59
+ config: eu
60
+ split: validation
61
+ args:
62
+ language: eu
63
+ metrics:
64
+ - name: Dev WER
65
+ type: wer
66
+ value: 4.17
67
+ ---
68
+
69
+ # HiTZ/Aholab's Basque Speech-to-Text model Conformer-Transducer
70
+ ## Model Description
71
+
72
+ <style>
73
+ img {
74
+ display: inline;
75
+ }
76
+ </style>
77
+
78
+ | [![Model architecture](https://img.shields.io/badge/Model_Arch-Conformer--CTC-lightgrey#model-badge)](#model-architecture)
79
+ | [![Model size](https://img.shields.io/badge/Params-121M-lightgrey#model-badge)](#model-architecture)
80
+ | [![Language](https://img.shields.io/badge/Language-eu-lightgrey#model-badge)](#datasets)
81
+
82
+ This model transcribes speech in lowercase Basque alphabet including spaces, and was trained on a composite dataset comprising of 548 hours of Basque speech. The model was fine-tuned from a pre-trained Spanish [stt_es_conformer_transducer_large](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_es_conformer_transducer_large) model using the [Nvidia NeMo](https://github.com/NVIDIA/NeMo) toolkit. It is an autoregressive "large" variant of Conformer, with around 119 million parameters.
83
+ See the [model architecture](#model-architecture) section and [NeMo documentation](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/models.html#conformer-transducer) for complete architecture details.
84
+
85
+ ## Usage
86
+ To train, fine-tune or play with the model you will need to install [NVIDIA NeMo](https://github.com/NVIDIA/NeMo). We recommend you install it after you've installed latest PyTorch version.
87
+
88
+ ```bash
89
+ pip install nemo_toolkit['all']
90
+ ```
91
+
92
+ ### Transcribing using Python
93
+ Clone repository to download the model:
94
+
95
+ ```bash
96
+ git clone https://huggingface.co/asierhv/stt_eu_conformer_transducer_large
97
+ ```
98
+
99
+ Given `NEMO_MODEL_FILEPATH` is the path that points to the downloaded `stt_eu_conformer_transducer_large.nemo` file.
100
+
101
+ ```python
102
+ import nemo.collections.asr as nemo_asr
103
+
104
+ # Load the model
105
+ asr_model = nemo_asr.models.EncDecRNNTBPEModel.restore_from(NEMO_MODEL_FILEPATH)
106
+
107
+ # Create a list pointing to the audio files
108
+ audio = ["audio_1.wav","audio_2.wav", ..., "audio_n.wav"]
109
+
110
+ # Fix the batch_size to whatever number suits your purpouse
111
+ batch_size = 8
112
+
113
+ # Transcribe the audio files
114
+ transcriptions = asr_model.transcribe(audio=audio, batch_size=batch_size)
115
+
116
+ # Visualize the transcriptions
117
+ print(transcriptions)
118
+ ```
119
+
120
+ ## Input
121
+ This model accepts 16000 kHz Mono-channel Audio (wav files) as input.
122
+
123
+ ## Output
124
+ This model provides transcribed speech as a string for a given audio sample.
125
+
126
+ ## Model Architecture
127
+ Conformer-Transducer model is an autoregressive variant of Conformer model [1] for Automatic Speech Recognition which uses Transducer loss/decoding instead of CTC loss. You may find more info on the detail of this model here: [Conformer-Transducer Model](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/models.html#conformer-transducer).
128
+
129
+ ## Training
130
+ ### Data preparation
131
+ This model has been trained on a composite dataset comprising 548 hours of Basque speech that contains:
132
+ - A processed subset of the `validated` split of the basque version of the public dataset [Mozilla Common Voice 16.1](https://huggingface.co/datasets/mozilla-foundation/common_voice_16_1): We have processed the `validated` split, which originally contains the `train`, `dev` and `test` splits, to create a subset free of sentences equal to the ones that are in the `test` split, to avoid leakage.
133
+ - The `train_clean` split of the basque version of the public dataset [Basque Parliament](https://huggingface.co/datasets/gttsehu/basque_parliament_1)
134
+ - A processed subset of the basque version of the public dataset [OpenSLR](https://huggingface.co/datasets/openslr#slr76-crowdsourced-high-quality-basque-speech-data-set): This subset has been cleaned from numerical characters and acronyms.
135
+
136
+ The composite dataset for training has been precisely cleaned from any sentence that equals the ones in the `test` datasets where the WER metrics will be computed.
137
+
138
+ ### Training procedure
139
+ This model was trained starting from the pre-trained Spanish model [stt_es_conformer_transducer_large](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo/models/stt_es_conformer_transducer_large) over several hundred of epochs in a GPU device, using the NeMo toolkit [3]
140
+ The tokenizer for these model was built using the text transcripts of the composite train dataset with this [script](https://github.com/NVIDIA/NeMo/blob/main/scripts/tokenizers/process_asr_text_tokenizer.py), with a total of 256 basque language tokens.
141
+
142
+ ## Performance
143
+ Performances of the ASR models are reported in terms of Word Error Rate (WER%) with greedy decoding in the following table.
144
+ | Tokenizer | Vocabulary Size | MCV 16.1 Test | Basque Parliament Test | Basque Parliament Dev | Train Dataset |
145
+ |-----------------------|-----------------|---------------|------------------------|-----------------------|------------------------------|
146
+ | SentencePiece Unigram | 256 | 2.79 | 3.83 | 4.17 | Composite Dataset (548 h) |
147
+
148
+ ## Limitations
149
+ Since this model was trained on almost publicly available speech datasets, the performance of this model might degrade for speech which includes technical terms, or vernacular that the model has not been trained on. The model might also perform worse for accented speech.
150
+
151
+ # Aditional Information
152
+ ## Author
153
+ HiTZ Basque Center for Language Technology - Aholab Signal Processing Laboratory, University of the Basque Country UPV/EHU.
154
+
155
+ ## Copyright
156
+ Copyright (c) 2024 HiTZ Basque Center for Language Technology - Aholab Signal Processing Laboratory, University of the Basque Country UPV/EHU.
157
+
158
+ ## Licensing Information
159
+ [Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/)
160
+
161
+ ## Funding
162
+ This project with reference 2022/TL22/00215335 has been parcially funded by the Ministerio de Transformación Digital and by the Plan de Recuperación, Transformación y Resiliencia – Funded by the European Union – NextGenerationEU [ILENIA](https://proyectoilenia.es/) and by the project [IkerGaitu](https://www.hitz.eus/iker-gaitu/) funded by the Basque Government.
163
+
164
+ ## References
165
+ - [1] [Conformer: Convolution-augmented Transformer for Speech Recognition](https://arxiv.org/abs/2005.08100)
166
+ - [2] [Google Sentencepiece Tokenizer](https://github.com/google/sentencepiece)
167
+ - [3] [NVIDIA NeMo Toolkit](https://github.com/NVIDIA/NeMo)
168
+
169
+ ## Disclaimer
170
+ <details>
171
+ <summary>Click to expand</summary>
172
+ The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions.
173
+
174
+ When third parties, deploy or provide systems and/or services to other parties using any of these models (or using systems based on these models) or become users of the models, they should note that it is their responsibility to mitigate the risks arising from their use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.
175
+
176
+ In no event shall the owner and creator of the models (HiTZ Basque Center for Language Technology - Aholab Signal Processing Laboratory, University of the Basque Country UPV/EHU.) be liable for any results arising from the use made by third parties of these models.