Spaces:
Runtime error
Runtime error
File size: 2,700 Bytes
ee21b96 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
# Self-Training with Kaldi HMM Models
This folder contains recipes for self-training on pseudo phone transcripts and
decoding into phones or words with [kaldi](https://github.com/kaldi-asr/kaldi).
To start, download and install kaldi follow its instruction, and place this
folder in `path/to/kaldi/egs`.
## Training
Assuming the following has been prepared:
- `w2v_dir`: contains features `{train,valid}.{npy,lengths}`, real transcripts `{train,valid}.${label}`, and dict `dict.${label}.txt`
- `lab_dir`: contains pseudo labels `{train,valid}.txt`
- `arpa_lm`: Arpa-format n-gram phone LM for decoding
- `arpa_lm_bin`: Arpa-format n-gram phone LM for unsupervised model selection to be used with KenLM
Set these variables in `train.sh`, as well as `out_dir`, the output directory,
and then run it.
The output will be:
```
==== WER w.r.t. real transcript (select based on unsupervised metric)
INFO:root:./out/exp/mono/decode_valid/scoring/14.0.0.tra.txt: score 0.9178 wer 28.71% lm_ppl 24.4500 gt_wer 25.57%
INFO:root:./out/exp/tri1/decode_valid/scoring/17.1.0.tra.txt: score 0.9257 wer 26.99% lm_ppl 30.8494 gt_wer 21.90%
INFO:root:./out/exp/tri2b/decode_valid/scoring/8.0.0.tra.txt: score 0.7506 wer 23.15% lm_ppl 25.5944 gt_wer 15.78%
```
where `wer` is the word eror rate with respect to the pseudo label, `gt_wer` to
the ground truth label, `lm_ppl` the language model perplexity of HMM prediced
transcripts, and `score` is the unsupervised metric for model selection. We
choose the model and the LM parameter of the one with the lowest score. In the
example above, it is `tri2b`, `8.0.0`.
## Decoding into Phones
In `decode_phone.sh`, set `out_dir` the same as used in `train.sh`, set
`dec_exp` and `dec_lmparam` to the selected model and LM parameter (e.g.
`tri2b` and `8.0.0` in the above example). `dec_script` needs to be set
according to `dec_exp`: for mono/tri1/tri2b, use `decode.sh`; for tri3b, use
`decode_fmllr.sh`.
The output will be saved at `out_dir/dec_data`
## Decoding into Words
`decode_word_step1.sh` prepares WFSTs for word decoding. Besides the variables
mentioned above, set
- `wrd_arpa_lm`: Arpa-format n-gram word LM for decoding
- `wrd_arpa_lm_bin`: Arpa-format n-gram word LM for unsupervised model selection
`decode_word_step1.sh` decodes the `train` and `valid` split into word and runs
unsupervised model selection using the `valid` split. The output is like:
```
INFO:root:./out/exp/tri2b/decodeword_valid/scoring/17.0.0.tra.txt: score 1.8693 wer 24.97% lm_ppl 1785.5333 gt_wer 31.45%
```
After determining the LM parameter (`17.0.0` in the example above), set it in
`decode_word_step2.sh` and run it. The output will be saved at
`out_dir/dec_data_word`.
|