artitsu's picture
Update README.md
e79ca99
metadata
license: cc-by-nc-4.0
language:
  - th
metrics:
  - cer
  - wer
library_name: espnet
pipeline_tag: automatic-speech-recognition

Model Card for Model ID

This is the baseline model of Khummuang in Thai-dialect corpus.

The training recipe was based on wsj recipe in espnet.

Model Description

This model is Hybrid CTC/Attention model with pre-trained HuBERT encoder.

Evaluation

For evaluation, the metrics are CER and WER. before WER evaluation, transcriptions were re-tokenized using newmm tokenizer in PyThaiNLP

In this reposirity, we also provide the vocabulary for building the newmm tokenizer using this script:

from pythainlp import Tokenizer

def get_tokenizer(vocab):

    custom_vocab = set(vocab)
    custom_tokenizer = Tokenizer(custom_vocab, engine='newmm')
    return custom_tokenizer

with open(<vocab_path>,'r',encoding='utf-8') as f:
        vocab = []
        for line in f.readlines():
            vocab.append(line.strip())

custom_tokenizer = get_tokenizer(vocab)

tokenized_sentence_list = custom_tokenizer.word_tokenize(<your_sentence>)
Micro CER Macro CER Survival CER E-commerce WER Micro WER Macro WER Survival WER E-commerce WER
5.35 5.65 6.29 5.02 7.53 8.73 11.38 6.09

Paper

Thai Dialect Corpus and Transfer-based Curriculum Learning Investigation for Dialect Automatic Speech Recognition

@inproceedings{suwanbandit23_interspeech,
  author={Artit Suwanbandit and Burin Naowarat and Orathai Sangpetch and Ekapol Chuangsuwanich},
  title={{Thai Dialect Corpus and Transfer-based Curriculum Learning Investigation for Dialect Automatic Speech Recognition}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
  pages={4069--4073},
  doi={10.21437/Interspeech.2023-1828}
}