Package: kaldi-active-grammar
Kaldi Active GrammarPython Kaldi speech recognition with grammars that can be set active/inactive dynamically at decode-timePython package developed to enable context-based command & control of computer applications, as in theDragonflyspeech recognition framework, using theKaldiautomatic speech recognition engine.Normally, Kaldi decoding graphs aremonolithic, requireexpensive up-front off-linecompilation, and arestatic during decoding. Kaldi's new grammar framework allowsmultiple independentgrammars with nonterminals, to be compiled separately andstitched together dynamicallyat decode-time, but all the grammars arealways activeand capable of being recognized.This project extends that to allow each grammar/rule to beindependently markedas active/inactivedynamicallyon aper-utterancebasis (set at the beginning of each utterance). Dragonfly is then capable of activatingonly the appropriate grammars for the current environment, resulting in increased accuracy due to fewer possible recognitions. Furthermore, the dictation grammar can besharedbetween all the command grammars, which can becompiled quicklywithout needing to include large-vocabulary dictation directly.See theChangelogfor the latest updates.FeaturesBinaries:The Python packageincludes all necessary binariesfor decoding onWindows/Linux/MacOS. Available onPyPI.Binaries are generated from myfork of Kaldi, which is only intended to be used by kaldi-active-grammar directly, and not as a stand-alone library.Pre-trained model:A compatiblegeneral English Kaldi nnet3 chain modelis trained on~3000hours of open audio. Available underproject releases.Model info and comparisonImproved models are under development.Plain dictation:Do you just want to recognize plain dictation? Seems kind of boring, but okay! There is aninterface for plain dictation(see below), using either your specifiedHCLG.fstfile, or KaldiAG's included pre-trained dictation model.Dragonfly/Caster:A compatiblebackend for Dragonflyis under development in thekaldibranch of my fork, and has been merged as of Dragonflyv0.15.0.See itsdocumentation, try out ademo, or use theloaderto run all normal dragonfly scripts.You can try it out easily on Windows using asimple no-install package: seeGetting Startedbelow.Casteris supported as of KaldiAGv0.6.0and Dragonflyv0.16.1.Bootstrappedsince v0.2: development of KaldiAG is done entirely using KaldiAG.Demo VideoDonations are appreciated to encourage development.Related Repositoriesdaanzu/kaldi-grammar-simpledaanzu/speech-training-recorderdaanzu/dragonfly_daanzu_toolskmdouglass/caster-kaldi: Docker image to run KaldiAG + Dragonfly + Caster inside a container on Linux, using the host's microphone.Getting StartedWant to get startedquickly & easily on Windows?
Available underproject releases:kaldi-dragonfly-winpython: A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run!kaldi-dragonfly-winpython-dev: [more recent development version] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run!kaldi-caster-winpython-dev: [more recent development version] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run!Otherwise...SetupRequirements:Python 3.6+;64-bit required!OS: Windows/Linux/MacOS all supportedOnly supports Kaldi left-biphone models, specificallynnet3 chainmodels, with specific modifications~1GB+ disk space for model plus temporary storage and cache, depending on your grammar complexity~1GB+ RAM for model and grammars, depending on your model and grammar complexityInstallation:Download compatible generic English Kaldi nnet3 chain model fromproject releases. Unzip the model and pass the directory path to kaldi-active-grammar constructor.Or use your own model. Standard Kaldi models must be converted to be usable. Conversion can be performed automatically, but this hasn't been fully implemented yet.Install Python package, which includes necessary Kaldi binaries:The easy way to use kaldi-active-grammar is as a backend to dragonfly, which makes it easy to define grammars and resultant actions.For this, simply runpip install 'dragonfly2[kaldi]'to install all necessary packages. See thedragonfly documentation for details on installation, plus how to define grammars and actions.Alternatively, if you only want to use it directly (via a more low level interface), you can just runpip install kaldi-active-grammarTo support automatic generation of pronunciations for unknown words (not in the lexicon), you have two choices:Local generation: Install theg2p_enpackage withpip install 'kaldi-active-grammar[g2p_en]'The necessary data files are now included in the latest speech models I released withv3.0.0.Online/cloud generation: Install therequestspackage withpip install 'kaldi-active-grammar[online]'ANDpassallow_online_pronunciations=TruetoCompiler.add_word()orModel.add_word()If both are available, the former is preferentially used.TroubleshootingErrors installingMake sure you're using a 64-bit Python.You should install viapip install kaldi-active-grammar(directly or indirectly),notpython setup.py install, in order to get the required binaries.Update yourpip(to at least19.0+) by executingpython -m pip install --upgrade pip, to support the required python binary wheel package.Errors runningWindows:The code execution cannot proceed because VCRUNTIME140.dll was not found.(or similar)You must install the VC2017+ redistributable from Microsoft:download page,direct link. (This is usually already installed globally by other programs.)Try deleting the Kaldi model.tmpdirectory, and re-running.Try deleting the Kaldi model directory itself, re-downloading and/or re-extracting it, and re-running. (Note: You may want to make a copy of youruser_lexicon.txtfile before deleting, to put in the new model directory.)For reporting issues, try running withimport logging; logging.basicConfig(level=1)at the top of your main/loader file to enable full debugging logging.DocumentationFormal documentation is somewhat lacking currently. To see example usage, examine:Plain dictation interface: Set up recognizer for plain dictation; perform decoding on givenwavfile.Full example: Set up grammar compiler & decoder; set up a rule; perform decoding on live, real-time audio from microphone.Backend for Dragonfly: Many advanced features and complex interactions.The KaldiAG API is fairly low level, but basically: you define a set of grammar rules, then send in audio data, along with a bit mask of which rules are active at the beginning of each utterance, and receive back the recognized rule and text. The easy way is to go through Dragonfly, which makes it easy to define the rules, contexts, and actions.BuildingRecommendation: use the binary wheels distributed for all major platforms.Significant work has gone into allowing you to avoid the many repo/dependency downloads, GBs of disk space, and vCPU-hours needed for building from scratch.They are built in public by automated Continuous Integration run on GitHub Actions:see manifest.Alternatively, to build for use locally:Linux/MacOS:python -m pip install -r requirements-build.txtpython setup.py bdist_wheel(seeCMakeLists.txtfor details)Windows:Less easily generally automatedYou can follow the steps for Continuous Integration run on GitHub Actions: see thebuild-windowssection ofthe manifest.Note: the project (and python wheel) is built from a duorepo (2 separate repos used together):This repo, containing the external interface and higher-level logic, written in Python.My fork of Kaldi, containing the lower-level code, written in C++.ContributingIssues, suggestions, and feature requests are welcome & encouraged. Pull requests are considered, but project structure is in flux.Donations are appreciated to encourage development.AuthorDavid Zurow (@daanzu)LicenseThis project is licensed under the GNU Affero General Public License v3 (AGPL-3.0-or-later). See theLICENSE.txt filefor details. If this license is problematic for you, please contact me.AcknowledgmentsBased on and including code fromKaldi ASR, under the Apache-2.0 license.Code fromOpenFSTandOpenFST port for Windows, under the Apache-2.0 license.Intel Math Kernel Library, copyright (c) 2018 Intel Corporation, under theIntel Simplified Software License, currently only used for Windows build.