Package: deepasr
DeepAsrDeepAsr is an open-source & Keras (Tensorflow) implementation of end-to-end Automatic Speech Recognition (ASR) engine and it supports multiple Speech Recognition architectures.Supported Asr Architectures:Baidu's Deep Speech 2DeepAsrNetwork1Using DeepAsr you can:perform speech-to-text using pre-trained modelstune pre-trained models to your needscreate new models on your ownDeepAsr key features:Multi GPU support: You can do much more like distribute the training using theStrategy, or experiment withmixed precisionpolicy.CuDNN support: Model usingCuDNNLSTMimplementation by NVIDIA Developers. CPU devices is also supported.DataGenerator: The feature extraction during model training for large the data.InstallationYou can use pip:pipinstalldeepasrGetting startedThe speech recognition is a tough task. You don't need to know all details to use one of the pretrained models.
However it's worth to understand conceptional crucial components:Input: Audio files (WAV or FLAC) with mono 16-bit 16 kHz (up to 5 seconds)FeaturesExtractor: Convert audio files using MFCC Features or SpectrogramModel: CTC model defined inKeras(references:[1],[2])Decoder: Greedy or BeamSearch algorithms with the language model support decode a sequence of probabilities using AlphabetDataGenerator: Stream data to the model via generatorCallbacks: Set of functions monitoring the trainingimportnumpyasnpimportpandasaspdimporttensorflowastfimportdeepasrasasr# get CTCPipelinedefget_config(feature_type:str='spectrogram',multi_gpu:bool=False):# audio feature extractorfeatures_extractor=asr.features.preprocess(feature_type=feature_type,features_num=161,samplerate=16000,winlen=0.02,winstep=0.025,winfunc=np.hanning)# input label encoderalphabet_en=asr.vocab.Alphabet(lang='en')# training modelmodel=asr.model.get_deepspeech2(input_dim=161,output_dim=29,is_mixed_precision=True)# model optimizeroptimizer=tf.keras.optimizers.Adam(lr=1e-4,beta_1=0.9,beta_2=0.999,epsilon=1e-8)# output label deocderdecoder=asr.decoder.GreedyDecoder()# decoder = asr.decoder.BeamSearchDecoder(beam_width=100, top_paths=1)# CTCPipelinepipeline=asr.pipeline.ctc_pipeline.CTCPipeline(alphabet=alphabet_en,features_extractor=features_extractor,model=model,optimizer=optimizer,decoder=decoder,sample_rate=16000,mono=True,multi_gpu=multi_gpu)returnpipelinetrain_data=pd.read_csv('train_data.csv')pipeline=get_config(feature_type='fbank',multi_gpu=False)# train asr modelhistory=pipeline.fit(train_dataset=train_data,batch_size=128,epochs=500)# history = pipeline.fit_generator(train_dataset = train_data, batch_size=32, epochs=500)pipeline.save('./checkpoint')Loaded pre-trained model has all components. The prediction can be invoked just by calling pipline.predict().importpandasaspdimportdeepasrasasrimportnumpyasnptest_data=pd.read_csv('test_data.csv')# get testing audio and transcript from datasetindex=np.random.randint(test_data.shape[0])data=test_data.iloc[index]test_file=data[0]test_transcript=data[1]# Test Audio fileprint("Audio File:",test_file)# Test Transcriptprint("Audio Transcript:",test_transcript)print("Transcript length:",len(test_transcript))pipeline=asr.pipeline.load('./checkpoint')print("Prediction",pipeline.predict(test_file))ReferencesThe fundamental repositories:Baidu -DeepSpeech2 - A PaddlePaddle implementation of DeepSpeech2 architecture for ASRNVIDIA -Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLPTensorFlow -The implementation of DeepSpeech2 modelMozilla -DeepSpeech - A TensorFlow implementation of Baidu's DeepSpeech architectureEspnet -End-to-End Speech Processing ToolkitAutomatic Speech Recognition -Distill the Automatic Speech Recognition researchPython Speech Features -Speech features for ASR including MFCCs and filterbank energies