---
tags:
- generated_from_keras_callback
model-index:
- name: wav2vec2-xls-r-300m-mixed
  results: []
---

<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-300m-mixed

Finetuned https://huggingface.co/facebook/wav2vec2-xls-r-300m on https://github.com/huseinzol05/malaya-speech/tree/master/data/mixed-stt

This model was finetuned on 3 languages,

1. Malay
2. Singlish
3. Mandarin

**This model trained on a single RTX 3090 Ti 24GB VRAM, provided by https://mesolitica.com/**. 

## Evaluation set

Evaluation set from https://github.com/huseinzol05/malaya-speech/tree/master/pretrained-model/prepare-stt with sizes,

```
len(malay), len(singlish), len(mandarin)
-> (765, 3579, 614)
```

It achieves the following results on the evaluation set based on [evaluate-gpu.ipynb](evaluate-gpu.ipynb):

Mixed evaluation,

```
CER: 0.0481054244857041
WER: 0.1322198446007387
CER with LM: 0.041196586938584696
WER with LM: 0.09880169127621556
```

Malay evaluation,

```
CER: 0.051636391937588406
WER: 0.19561999547293663
CER with LM: 0.03917689630621449
WER with LM: 0.12710746406824835
```

Singlish evaluation,

```
CER: 0.0494915200071987
WER: 0.12763802881676573
CER with LM: 0.04271234986432335
WER with LM: 0.09677160640413336
```

Mandarin evaluation,

```
CER: 0.035626554824269824
WER: 0.07993515937860181
CER with LM: 0.03487760945087219
WER with LM: 0.07536807168546154
```

Language model from https://huggingface.co/huseinzol05/language-model-bahasa-manglish-combined