Hugging Face's logo

espnet
/
WavLabLM-MK-40k

ESPnet
102 languages
audio
self-supervised-learning
speech-recognition
Model card Files Community
WavLabLM-MK-40k
File size: 3,013 Bytes 
09ffb0d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
be1c3ec
09ffb0d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
 
---
tags:
- espnet
- audio
- self-supervised-learning
- speech-recognition
multilinguality:
  - multilingual
task_categories:
  - automatic-speech-recognition
language:
  - afr
  - amh
  - ara
  - asm
  - ast
  - azj
  - bel
  - ben
  - bos
  - cat
  - ceb
  - cmn
  - ces
  - cym
  - dan
  - deu
  - ell
  - eng
  - spa
  - est
  - fas
  - ful
  - fin
  - tgl
  - fra
  - gle
  - glg
  - guj
  - hau
  - heb
  - hin
  - hrv
  - hun
  - hye
  - ind
  - ibo
  - isl
  - ita
  - jpn
  - jav
  - kat
  - kam
  - kea
  - kaz
  - khm
  - kan
  - kor
  - ckb
  - kir
  - ltz
  - lug
  - lin
  - lao
  - lit
  - luo
  - lav
  - mri
  - mkd
  - mal
  - mon
  - mar
  - msa
  - mlt
  - mya
  - nob
  - npi
  - nld
  - nso
  - nya
  - oci
  - orm
  - ory
  - pan
  - pol
  - pus
  - por
  - ron
  - rus
  - bul
  - snd
  - slk
  - slv
  - sna
  - som
  - srp
  - swe
  - swh
  - tam
  - tel
  - tgk
  - tha
  - tur
  - ukr
  - umb
  - urd
  - uzb
  - vie
  - wol
  - xho
  - yor
  - yue
  - zul
datasets:
- fleurs
- babel
- voxpopuli
- commonvoice
license: cc-by-4.0
---

## WavLabLM-MK 40k

[Paper](https://arxiv.org/abs/2309.15317)

This model was trained by [William Chen](https://wanchichen.github.io/) using ESPNet2's SSL recipe in [espnet](https://github.com/espnet/espnet/).
WavLabLM is an self-supervised audio encoder pre-trained on 40,000 hours of multilingual data across 136 languages. This specific variant, WavLabLM-MK, uses a K-means model trained on multilingual data for the quantization. 
It is used as the base model for [WavLabLM-MS](https://huggingface.co/espnet/WavLabLM-MS-40k), which is better tuned for non-European languages.


```BibTex
@misc{chen2023joint,
      title={Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learning}, 
      author={William Chen and Jiatong Shi and Brian Yan and Dan Berrebbi and Wangyou Zhang and Yifan Peng and Xuankai Chang and Soumi Maiti and Shinji Watanabe},
      year={2023},
      eprint={2309.15317},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```


### Citing ESPnet

```BibTex
@inproceedings{watanabe2018espnet,
  author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
  title={{ESPnet}: End-to-End Speech Processing Toolkit},
  year={2018},
  booktitle={Proceedings of Interspeech},
  pages={2207--2211},
  doi={10.21437/Interspeech.2018-1456},
  url={http://dx.doi.org/10.21437/Interspeech.2018-1456}
}


```

or arXiv:

```bibtex
@misc{watanabe2018espnet,
  title={ESPnet: End-to-End Speech Processing Toolkit}, 
  author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
  year={2018},
  eprint={1804.00015},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
```