Question: How large is the dataset used for evaluation?
Evidence: We evaluate our approach on the large vocabulary task of the Wall Street Journal (WSJ) dataset BIBREF25 , which contains 80 hours of clean read speech, and Librispeech BIBREF26 , which contains 1000 hours with separate train/dev/test splits for clean and noisy speech. Each dataset comes with official textual data to train language models, which contain 37 million tokens for WSJ, 800 million tokens for Librispeech. Our language models are trained separately for each dataset on the official text data only. These datasets were chosen to study the impact of the different components of our system at different scales of training data and in different recording conditions.
The models are evaluated in Word Error Rate (WER). Our experiments use the open source codes of wav2letter for the acoustic model, and fairseq for the language model. More details on the experimental setup are given below.
Baseline Our baseline for each dataset follows BIBREF11 . It uses the same convolutional acoustic model as our approach but a mel-filterbanks front-end and a 4-gram language model.
Training/test splits On WSJ, models are trained on si284. nov93dev is used for validation and nov92 for test. On Librispeech, we train on the concatenation of train-clean and train-other. The validation set is dev-clean when testing on test-clean, and dev-other when testing on test-other.
Acoustic model architecture The architecture for the convolutional acoustic model is the "high dropout" model from BIBREF11 for Librispeech, which has 19 layers in addition to the front-end (mel-filterbanks for the baseline, or the learnable front-end for our approach). On WSJ, we use the lighter version used in BIBREF17 , which has 17 layers. Dropout is applied at each layer after the front-end, following BIBREF20 . The learnable front-end uses 40 or 80 filters. Language model architecture As described in Section SECREF8 , we use the GCNN-14B model of BIBREF0 with dropout at each convolutional and linear layer on both WSJ and Librispeech. We keep all the words (162K) in WSJ training corpus. For Librispeech, we only use the most frequent 200K tokens (out of 900K).
Hyperparameter tuning The acoustic models are trained following BIBREF11 , BIBREF17 , using SGD with a decreasing learning rate, weight normalization and gradient clipping at 0.2 and a momentum of 0.9. The language models are trained with Nesterov accelerated gradient BIBREF27 . Following BIBREF0 , we also use weight normalization and gradient clipping.
The parameters of the beam search (see Section SECREF9 ) INLINEFORM0 , INLINEFORM1 and INLINEFORM2 are tuned on the validation set with a beam size of 2500 and a beam score of 26 for computational efficiency. Once INLINEFORM3 are chosen, the test WER is computed with a beam size of 3000 and a beam score of 50.
We evaluate our approach on the large vocabulary task of the Wall Street Journal (WSJ) dataset [24], which contains 80 hours of clean read speech, and Librispeech [25], which contains 1000 hours with separate train/dev/test splits for clean and noisy speech.
Answer: They evaluate their approach on the large vocabulary task of the Wall Street Journal (WSJ) dataset [24], which contains 80 hours of clean read speech, and Librispeech which contains 1000 hours with separate train/dev/test splits for clean and noisy speech.