Package: wavencoder
WavEncoderWavEncoder is a Python library for encoding audio signals, transforms for audio augmentation, and training audio classification models with PyTorch backend.Package ContentsLayersModelsTransformsTrainer and utilsAttentionDotSoftAdditiveMultiplicativeSincNet layerTime Delay Neural Network(TDNN)PreTrainedwav2vecwav2vec2(base, large, xlsr53)SincNetRawNetBaseline1DCNNLSTM ClassifierLSTM Attention ClassifierNoise(Environmet/Gaussian White Noise)Speed ChangePadCropClipReverberationTimeShiftTimeMaskFrequencyMaskClassification TrainerClassification TestingDownload Noise DatasetDownload Impulse Response DatasetWav Models to be addedwav2vec[1]wav2vec2[2]SincNet[3]PASE[4]MockingJay[5]RawNet[6]GaborNet[7]LEAF[8]CNN-1DCNN-LSTMCNN-LSTM-AttnCheck theDemo Colab Notebook.InstallationUse the package managerpipto install wavencoder.pipinstallwavencoderUsageImport pretrained encoder, baseline models and classifiersimporttorchimportwavencoderx=torch.randn(1,16000)# [1, 16000]encoder=wavencoder.models.Wav2Vec(pretrained=True)z=encoder(x)# [1, 512, 98]classifier=wavencoder.models.LSTM_Attn_Classifier(512,64,2,return_attn_weights=True,attn_type='soft')y_hat,attn_weights=classifier(z)# [1, 2], [1, 98]Use wavencoder with PyTorch Sequential or class modulesimporttorchimporttorch.nnasnnimportwavencodermodel=nn.Sequential(wavencoder.models.Wav2Vec(),wavencoder.models.LSTM_Attn_Classifier(512,64,2,return_attn_weights=True,attn_type='soft'))x=torch.randn(1,16000)# [1, 16000]y_hat,attn_weights=model(x)# [1, 2], [1, 98]importtorchimporttorch.nnasnnimportwavencoderclassAudioClassifier(nn.Module):def__init__(self):super(AudioClassifier,self).__init__()self.encoder=wavencoder.models.Wav2Vec(pretrained=True)self.classifier=nn.Linear(512,2)defforward(self,x):z=self.encoder(x)z=torch.mean(z,dim=2)out=self.classifier(z)returnoutmodel=AudioClassifier()x=torch.randn(1,16000)# [1, 16000]y_hat=model(x)# [1, 2]Train the encoder-classifier modelsfromwavencoder.modelsimportWav2Vec,LSTM_Attn_Classifierfromwavencoder.trainerimporttrain,test_evaluate_classifier,test_predict_classifiermodel=nn.Sequential(Wav2Vec(pretrained=False),LSTM_Attn_Classifier(512,64,2))trainloader=...valloader=...testloader=...trained_model,train_dict=train(model,trainloader,valloader,n_epochs=20)test_prediction_dict=test_predict_classifier(trained_model,testloader)Add Transforms to your DataLoader for Augmentation/Processing the wav signalfromwavencoder.transformsimportCompose,AdditiveNoise,SpeedChange,Clipping,PadCrop,Reverberationaudio,_=torchaudio.load('test.wav')transforms=Compose([AdditiveNoise('path-to-noise-folder',p=0.5,snr_levels=[5,10,15],p=0.5),SpeedChange(factor_range=(-0.5,0.0),p=0.5),Clipping(p=0.5),PadCrop(48000,crop_position='random',pad_position='random')])transformed_audio=transforms(audio)ContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.Please make sure to update tests as appropriate.LicenseMITReferencePaperCode[1]Wav2Vec: Unsupervised Pre-training for Speech RecognitionGitHub[2]Wav2vec 2.0: Learning the structure of speech from raw audioGitHub[3]Speaker Recognition from Raw Waveform with SincNetGitHub[4]Learning Problem-agnostic Speech Representations from Multiple Self-supervised TasksGitHub[5]Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer EncodersGitHub[6]Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw WaveformsGitHub