README.md

---
license: cc-by-nc-4.0
language:
- th
metrics:
- cer
- wer
library_name: espnet
pipeline_tag: automatic-speech-recognition
---
# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This is the baseline model of Khummuang in [Thai-dialect corpus](https://github.com/SLSCU/thai-dialect-corpus).

The training recipe was based on wsj recipe in [espnet](https://github.com/espnet/espnet/).



### Model Description

<!-- Provide a longer summary of what this model is. -->

This model is Hybrid CTC/Attention model with pre-trained HuBERT encoder.



## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

For evaluation, the metrics are CER and WER. before WER evaluation, transcriptions were re-tokenized using newmm tokenizer in [PyThaiNLP](https://github.com/PyThaiNLP/pythainlp)

In this reposirity, we also provide the vocabulary for building the newmm tokenizer using this script:

```python
from pythainlp import Tokenizer

def get_tokenizer(vocab):

    custom_vocab = set(vocab)
    custom_tokenizer = Tokenizer(custom_vocab, engine='newmm')
    return custom_tokenizer

with open(<vocab_path>,'r',encoding='utf-8') as f:
        vocab = []
        for line in f.readlines():
            vocab.append(line.strip())

custom_tokenizer = get_tokenizer(vocab)

tokenized_sentence_list = custom_tokenizer.word_tokenize(<your_sentence>)
```

|Micro CER|Macro CER|Survival CER|E-commerce WER|Micro WER|Macro WER|Survival WER|E-commerce WER|
|---|---|---|---|---|---|---|---|
|5.35|5.65|6.29|5.02|7.53|8.73|11.38|6.09|
<!-- |8.08|11.51|17.39|5.63|12.18|16.65|25.58|7.72| -->
<!-- |18.17|22.38|31.01|13.75|31.74|37.68|50.54|24.82| -->

## Paper

[Thai Dialect Corpus and Transfer-based Curriculum Learning Investigation for Dialect Automatic Speech Recognition](https://www.isca-speech.org/archive/pdfs/interspeech_2023/suwanbandit23_interspeech.pdf)
```
@inproceedings{suwanbandit23_interspeech,
  author={Artit Suwanbandit and Burin Naowarat and Orathai Sangpetch and Ekapol Chuangsuwanich},
  title={{Thai Dialect Corpus and Transfer-based Curriculum Learning Investigation for Dialect Automatic Speech Recognition}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
  pages={4069--4073},
  doi={10.21437/Interspeech.2023-1828}
}
```