Hugging Face's logo

espnet
/
WavLabLM-MS-40k

ESPnet
audio
self-supervised-learning
speech-recognition
Model card Files Community
1
WavLabLM-MS-40k
File size: 3,075 Bytes 
bcbd17d
 
 
 
 
78a2f20
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
bcbd17d
 
 
 
 
 
 
 
 
 
6b2f84f
 
 
 
 
bcbd17d
d4cf80d
 
 
 
 
 
 
 
 
 
bcbd17d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
 
---
tags:
- espnet
- audio
- self-supervised-learning
- speech-recognition
multilinguality:
  - multilingual
task_categories:
  - automatic-speech-recognition
language:
  - afr
  - amh
  - ara
  - asm
  - ast
  - azj
  - bel
  - ben
  - bos
  - cat
  - ceb
  - cmn
  - ces
  - cym
  - dan
  - deu
  - ell
  - eng
  - spa
  - est
  - fas
  - ful
  - fin
  - tgl
  - fra
  - gle
  - glg
  - guj
  - hau
  - heb
  - hin
  - hrv
  - hun
  - hye
  - ind
  - ibo
  - isl
  - ita
  - jpn
  - jav
  - kat
  - kam
  - kea
  - kaz
  - khm
  - kan
  - kor
  - ckb
  - kir
  - ltz
  - lug
  - lin
  - lao
  - lit
  - luo
  - lav
  - mri
  - mkd
  - mal
  - mon
  - mar
  - msa
  - mlt
  - mya
  - nob
  - npi
  - nld
  - nso
  - nya
  - oci
  - orm
  - ory
  - pan
  - pol
  - pus
  - por
  - ron
  - rus
  - bul
  - snd
  - slk
  - slv
  - sna
  - som
  - srp
  - swe
  - swh
  - tam
  - tel
  - tgk
  - tha
  - tur
  - ukr
  - umb
  - urd
  - uzb
  - vie
  - wol
  - xho
  - yor
  - yue
  - zul
datasets:
- fleurs
- babel
- voxpopuli
- commonvoice
license: cc-by-4.0
---

## WavLabLM-MS 40k

[Paper](https://arxiv.org/abs/2309.15317)

This model was trained by [William Chen](https://wanchichen.github.io/) using ESPNet2's SSL recipe in [espnet](https://github.com/espnet/espnet/).
WavLabLM is an self-supervised audio encoder pre-trained on 40,000 hours of multilingual data across 136 languages. This specific variant, WavLabLM-MS, went through a second stage of pre-training on a balanced subset of the data to improve performance on lower-resource languages.
It achieves comparable performance to XLS-R 128 on the [ML-SUPERB Benchmark](https://arxiv.org/abs/2305.10615) with only 10% of the pre-training data.

```BibTex
@misc{chen2023joint,
      title={Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learning}, 
      author={William Chen and Jiatong Shi and Brian Yan and Dan Berrebbi and Wangyou Zhang and Yifan Peng and Xuankai Chang and Soumi Maiti and Shinji Watanabe},
      year={2023},
      eprint={2309.15317},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```


### Citing ESPnet

```BibTex
@inproceedings{watanabe2018espnet,
  author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
  title={{ESPnet}: End-to-End Speech Processing Toolkit},
  year={2018},
  booktitle={Proceedings of Interspeech},
  pages={2207--2211},
  doi={10.21437/Interspeech.2018-1456},
  url={http://dx.doi.org/10.21437/Interspeech.2018-1456}
}


```

or arXiv:

```bibtex
@misc{watanabe2018espnet,
  title={ESPnet: End-to-End Speech Processing Toolkit}, 
  author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
  year={2018},
  eprint={1804.00015},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
```