Introduction

This repo contains pre-trained model using https://github.com/k2-fsa/icefall/pull/213.

It is trained on train-clean-100 subset of the LibriSpeech dataset. Also, it uses the S subset from GigaSpeech as extra training data.

How to clone this repo

sudo apt-get install git-lfs
git clone https://huggingface.co/csukuangfj/icefall-asr-librispeech-100h-transducer-stateless-multi-datasets-bpe-500-2022-02-21

cd icefall-asr-librispeech-100h-transducer-stateless-multi-datasets-bpe-500-2022-02-21
git lfs pull

Catuion: You have to run git lfs pull. Otherwise, you will be SAD later.

The model in this repo is trained using the commit 2332ba312d7ce72f08c7bac1e3312f7e3dd722dc.

You can use

git clone https://github.com/k2-fsa/icefall
cd icefall
git checkout 2332ba312d7ce72f08c7bac1e3312f7e3dd722dc

to download icefall.

You can find the model information by visiting https://github.com/k2-fsa/icefall/blob/2332ba312d7ce72f08c7bac1e3312f7e3dd722dc/egs/librispeech/ASR/transducer_stateless_multi_datasets/train.py#L198.

In short, the encoder is a Conformer model with 8 heads, 12 encoder layers, 512-dim attention, 2048-dim feedforward; the decoder contains a 1024-dim embedding layer and a Conv1d with kernel size 2.

The decoder architecture is modified from Rnn-Transducer with Stateless Prediction Network. A Conv1d layer is placed right after the input embedding layer.

Description

This repo provides pre-trained transducer Conformer model for the LibriSpeech dataset using icefall. There are no RNNs in the decoder. The decoder is stateless and contains only an embedding layer and a Conv1d.

The commands for training are:

cd egs/librispeech/ASR/
./prepare.sh
./prepare_giga_speech.sh

export CUDA_VISIBLE_DEVICES="0,1"

./transducer_stateless_multi_datasets/train.py \
  --world-size 2 \
  --num-epochs 60 \
  --start-epoch 0 \
  --exp-dir transducer_stateless_multi_datasets/exp-100-2 \
  --full-libri 0 \
  --max-duration 300 \
  --lr-factor 1 \
  --bpe-model data/lang_bpe_500/bpe.model \
  --modified-transducer-prob 0.25
  --giga-prob 0.2

The tensorboard training log can be found at https://tensorboard.dev/experiment/qUEKzMnrTZmOz1EXPda9RA/

The command for decoding is:

epoch=57
avg=17

## greedy search
for epoch in 57; do
  for avg in 17; do
    for sym in 1 2 3; do
    ./transducer_stateless_multi_datasets/decode.py \
      --epoch $epoch \
      --avg $avg \
      --exp-dir transducer_stateless_multi_datasets/exp-100-2 \
      --bpe-model ./data/lang_bpe_500/bpe.model \
      --max-duration 100 \
      --context-size 2 \
      --max-sym-per-frame $sym
    done
  done
done

## modified beam search

epoch=57
avg=17
./transducer_stateless_multi_datasets/decode.py \
  --epoch $epoch \
  --avg $avg \
  --exp-dir transducer_stateless_multi_datasets/exp-100-2 \
  --bpe-model ./data/lang_bpe_500/bpe.model \
  --max-duration 100 \
  --context-size 2 \
  --decoding-method modified_beam_search \
  --beam-size 4

You can find the decoding log for the above command in this repo (in the folder log).

The WERs for the test datasets are

	test-clean	test-other	comment
greedy search (max sym per frame 1)	6.34	16.7	--epoch 57, --avg 17, --max-duration 100
greedy search (max sym per frame 2)	6.34	16.7	--epoch 57, --avg 17, --max-duration 100
greedy search (max sym per frame 3)	6.34	16.7	--epoch 57, --avg 17, --max-duration 100
modified beam search (beam size 4)	6.31	16.3	--epoch 57, --avg 17, --max-duration 100

File description

log, this directory contains the decoding log and decoding results
test_wavs, this directory contains wave files for testing the pre-trained model
data, this directory contains files generated by prepare.sh
exp, this directory contains only one file: preprained.pt

exp/pretrained.pt is generated by the following command:

./transducer_stateless_multi_datasets/export.py \
  --epoch 57 \
  --avg 17 \
  --bpe-model data/lang_bpe_500/bpe.model \
  --exp-dir transducer_stateless_multi_datasets/exp-full

HINT: To use pretrained.pt to compute the WER for test-clean and test-other, just do the following:

cp icefall-asr-librispeech-100h-transducer-stateless-multi-datasets-bpe-500-2022-02-21/exp/pretrained.pt \
  /path/to/icefall/egs/librispeech/ASR/transducer_stateless_multi_datasets/exp/epoch-999.pt

and pass --epoch 999 --avg 1 to transducer_stateless_multi_datasets/decode.py.