Package: tiramisu-asr
TiramisuASR :cake:Almost State-of-the-art Automatic Speech Recognition in Tensorflow 2TiramisuASR implements some speech recognition architectures such as CTC-based models (Deep Speech 2, etc.), RNN Transducer (Conformer, etc.). These models can be converted to TFLite to reduce memory and computation for deployment :smile:What's New?(10/10/2020) Update documents and upload package to pypi(10/6/2020) Changenlpaugversion to>=1.0.1(9/18/2020) Supportword-pieces(akasubwords) usingtensorflow-datasetsSupporttransducertflite greedy decoding (conversion and invocation)Distributed training usingtf.distribute.MirroredStrategy:yum: Supported ModelsCTCModel(End2end models using CTC Loss for training)Transducer Models(End2end models using RNNT Loss for training)Conformer Transducer(Reference:https://arxiv.org/abs/2005.08100)
Seeexamples/conformerSetup Environment and DatasetsInstall tensorflow:pip3 install -U tensorfloworpip3 install tf-nightly(for using tflite)Install packages (chooseoneof these options):Runpip3 install -U tiramisu-asrClone the repo and runpython3 setup.py installin the repo's directoryForsetting up datasets, seedatasetsFortraining, testing and usingCTC Models, run./scripts/install_ctc_decoders.shFortrainingTransducer Models, exportCUDA_HOMEand run./scripts/install_rnnt_loss.shMethodtiramisu_asr.utils.setup_environment()enablemixed_precisionif available.To enable XLA, runTF_XLA_FLAGS=--tf_xla_auto_jit=2 $python_train_scriptClean up:python3 setup.py clean --all(this will remove/buildcontents)TFLite ConvertionAfter converting to tflite, the tflite model is like a function that transforms directly from anaudio signaltounicode code points, then we can convert unicode points to string.Installtf-nightlyusingpip install tf-nightlyBuild a model with the same architecture as the trained model(if model has tflite argument, you must set it to True), then load the weights from trained model to the built modelLoadTFSpeechFeaturizerandTextFeaturizerto model using functionadd_featurizersConvert model's function to tflite as follows:func=model.make_tflite_function(greedy=True)# or Falseconcrete_func=func.get_concrete_function()converter=tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])converter.experimental_new_converter=Trueconverter.optimizations=[tf.lite.Optimize.DEFAULT]converter.target_spec.supported_ops=[tf.lite.OpsSet.TFLITE_BUILTINS,tf.lite.OpsSet.SELECT_TF_OPS]tflite_model=converter.convert()Save the converted tflite model as follows:ifnotos.path.exists(os.path.dirname(tflite_path)):os.makedirs(os.path.dirname(tflite_path))withopen(tflite_path,"wb")astflite_out:tflite_out.write(tflite_model)Then the.tflitemodel is ready to be deployedFeatures ExtractionSeefeatures_extractionAugmentationsSeeaugmentationsTraining & TestingExample YAML Config Structurespeech_config:...model_config:...decoder_config:...learning_config:augmentations:...dataset_config:train_paths:...eval_paths:...test_paths:...tfrecords_dir:...optimizer_config:...running_config:batch_size:8num_epochs:20outdir:...log_interval_steps:500Seeexamplesfor some predefined ASR models and resultsCorpus Sources and Pretrained ModelsFor pretrained models, go todriveEnglishNameSourceHoursLibriSpeechLibriSpeech970hCommon Voicehttps://commonvoice.mozilla.org1932hVietnameseNameSourceHoursVivoshttps://ailab.hcmus.edu.vn/vivos15hInfoRe Technology 1InfoRe1 (passwd: BroughtToYouByInfoRe)25hInfoRe Technology 2 (used in VLSP2019)InfoRe2 (passwd: BroughtToYouByInfoRe)415hGermanNameSourceHoursCommon Voicehttps://commonvoice.mozilla.org/750hReferences & CreditsNVIDIA OpenSeq2Seq Toolkithttps://github.com/noahchalifour/warp-transducerSequence Transduction with Recurrent Neural NetworkEnd-to-End Speech Processing Toolkit in PyTorch