Package: howl
HowlWake word detection modeling for Firefox Voice, supporting open datasets like Google Speech Commands and Mozilla Common Voice.Citation:@inproceedings{tang-etal-2020-howl,
    title = "Howl: A Deployed, Open-Source Wake Word Detection System",
    author = "Tang, Raphael and Lee, Jaejun and Razi, Afsaneh and Cambre, Julia and Bicking, Ian and Kaye, Jofish and Lin, Jimmy",
    booktitle = "Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)",
    month = nov,
    year = "2020",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.nlposs-1.9",
    doi = "10.18653/v1/2020.nlposs-1.9",
    pages = "61--65"
}Quickstart GuideInstall PyAudio andPyTorch 1.5+through your distribution's package system.Install Howl usingpippip install howlTo immediately use a pre-trained Howl model for inference, we provide theclientAPI. The following example (also found underexamples/hey_fire_fox.py) loads the "hey_fire_fox" pretrained model with a simple callback and starts the inference client.from howl.client import HowlClient

def hello_callback(detected_words):
    print("Detected: {}".format(detected_words))

client = HowlClient()
client.from_pretrained("hey_fire_fox", force_reload=False)
client.add_listener(hello_callback)
client.start().join()Training GuideInstallationgit clone https://github.com/castorini/howl && cd howlInstallPyTorchby following your platform-specific instructions.Install PyAudio and its dependencies through your distribution's package system.pip install -r requirements.txt -r requirements_training.txt(some apt packages might need to be installed)Preparing a DatasetIn the example that follows, we describe how to train a custom detector for the word, "fire."Download a supported data source. We recommendCommon Voicefor its breadth and free license.To provide alignment for the data, installMontreal Forced Aligner(MFA)
and download anEnglish pronunciation dictionary.Create a positive dataset containing the keyword:VOCAB='["fire"]'INFERENCE_SEQUENCE=[0]DATASET_PATH=data/fire-positivepython-mhowl_training.run.create_raw_dataset--negative-pct0-i~/path/to/common-voice--positive-pct100Create a negative dataset without the keyword:VOCAB='["fire"]'INFERENCE_SEQUENCE=[0]DATASET_PATH=data/fire-negativepython-mhowl_training.run.create_raw_dataset--negative-pct5-i~/path/to/common-voice--positive-pct0Generate some mock alignment for the negative set, where we don't care about alignment:DATASET_PATH=data/fire-negativepython-mhowl_training.run.attach_alignment--align-typestubUse MFA to generate alignment for the positive set:mfa_aligndata/fire-positive/audioeng.dictpretrained_models/english.zipoutput-folderAttach the MFA alignment to the positive dataset:DATASET_PATH=data/fire-positivepython-mhowl_training.run.attach_alignment--align-typemfa-ioutput-folderTraining and Running a ModelSource the relevant environment variables for training theres8model:source envs/res8.env.Train the model:python -m howl_training.run.train -i data/fire-negative data/fire-positive --model res8 --workspace workspaces/fire-res8.For the CLI demo, runpython -m howl_training.run.demo --model res8 --workspace workspaces/fire-res8.Pretrained Modelshowl-modelscontains workspaces with pretrained modelsTo get the latest models, simply rungit submodule update --init --recursivehey firefoxVOCAB='[" hey","fire","fox"]'INFERENCE_SEQUENCE=[0,1,2]INFERENCE_THRESHOLD=0NUM_MELS=40MAX_WINDOW_SIZE_SECONDS=0.5python-mhowl_training.run.demo--modelres8--workspacehowl-models/howl/hey-fire-foxReproducing Paper ResultsFirst, follow the installation instructions in the quickstart guide.Google Speech CommandsDownloadthe Google Speech Commands datasetand extract it.Source the appropriate environment variables:source envs/res8.envSet the dataset path to the root folder of the Speech Commands dataset:export DATASET_PATH=/path/to/datasetTrain theres8model:NUM_EPOCHS=20 MAX_WINDOW_SIZE_SECONDS=1 VOCAB='["yes","no","up","down","left","right","on","off","stop","go"]' BATCH_SIZE=64 LR_DECAY=0.8 LEARNING_RATE=0.01 python -m howl_training.run.pretrain_gsc --model res8Hey FirefoxDownloadthe Hey Firefox corpus, licensed under CC0, and extract it.Downloadour noise dataset, built from Microsoft SNSD and MUSAN, and extract it.Source the appropriate environment variables:source envs/res8.envSet the noise dataset path to the root folder:export NOISE_DATASET_PATH=/path/to/snsdSet the firefox dataset path to the root folder:export DATASET_PATH=/path/to/hey_firefoxTrain the model:LR_DECAY=0.98 VOCAB='[" hey","fire","fox"]' USE_NOISE_DATASET=True BATCH_SIZE=16 INFERENCE_THRESHOLD=0 NUM_EPOCHS=300 NUM_MELS=40 INFERENCE_SEQUENCE=[0,1,2] MAX_WINDOW_SIZE_SECONDS=0.5 python -m howl_training.run.train --model res8 --workspace workspaces/hey-ff-res8Hey SnipsDownloadhey snips datasetProcess the dataset to a format howl can loadVOCAB='["hey","snips"]'INFERENCE_SEQUENCE=[0,1]DATASET_PATH=data/hey-snipspython-mhowl_training.run.create_raw_dataset--dataset-type'hey-snips'-i~/path/to/hey_snips_datasetGenerate some mock alignment for the dataset, where we don't care about alignment:DATASET_PATH=data/hey-snipspython-mhowl_training.run.attach_alignment--align-typestubUse MFA to generate alignment for the dataset set:mfa_aligndata/hey-snips/audioeng.dictpretrained_models/english.zipoutput-folderAttach the MFA alignment to the dataset:DATASET_PATH=data/hey-snipspython-mhowl_training.run.attach_alignment--align-typemfa-ioutput-folderSource the appropriate environment variables:source envs/res8.envSet the noise dataset path to the root folder:export NOISE_DATASET_PATH=/path/to/snsdSet the noise dataset path to the root folder:export DATASET_PATH=/path/to/hey-snipsTrain the model:LR_DECAY=0.98 VOCAB='[" hey","snips"]' USE_NOISE_DATASET=True BATCH_SIZE=16 INFERENCE_THRESHOLD=0 NUM_EPOCHS=300 NUM_MELS=40 INFERENCE_SEQUENCE=[0,1] MAX_WINDOW_SIZE_SECONDS=0.5 python -m howl_training.run.train --model res8 --workspace workspaces/hey-snips-res8Generating dataset for Mycroft-precisehowl also provides a script for transforming howl dataset tomycroft-precisedatasetVOCAB='[" hey","fire","fox"]'INFERENCE_SEQUENCE=[0,1,2]python-mhowl_training.run.generate_precise_dataset--dataset-path/path/to/howl_datasetExperimentsTo verify the correctness of our implementation, we first train and evaluate our models on the Google Speech Commands dataset, for which there exists many known results. Next, we curate a wake word detection datasets and report our resulting model quality.For both experiments, we generate reports in excel format.experimentsfolder includes sample outputs from the for each experiment and corresponding workspaces can be foundherecommands_recognitionFor command recognition, we train the four different models (res8, LSTM, LAS encoder, MobileNetv2) to detect twelve different keywords: “yes”, “no”, “up”, “down”, “left”, “right”, “on”, “off”, “stop”, “go”, unknown, or silence.python-mhowl_training.run.eval_commands_recognition--num_iterationsn--dataset_path<path_to_gsc_datasets>word_detectionIn this experiment, we train our best commands recognition model, res8, forhey firefoxandhey snipsand evaluate them with different threashold.Two different performance reports are generated, one with the clean audio and one with audios with noisepython-mhowl_training.run.eval_wake_word_detection--num_modelsn--hop_size<numberbetween0and1>--exp_type<hey_firefox|hey_snips>--dataset_path"x"--noiseset_path"y"We also provide a script for generating ROC curve.exp_timestampcan be found from the reports generated from previous commandpython-mhowl_training.run.generate_roc--exp_timestamp<experimenttimestamp>--exp_type<hey_firefox|hey_snips>