Package: waveaugment
WaveAugmentWavAugment performs data augmentation on audio data.
The audio data is represented aspytorchtensors.It is particularly useful for speech data.
Among others, it implements the augmentations that we found to be most useful for self-supervised learning
(Data Augmenting Contrastive Learning of Speech Representations in the Time Domain, E. Kharitonov, M. Rivière, G. Synnaeve, L. Wolf, P.-E. Mazaré, M. Douze, E. Dupoux.[arxiv]):Pitch randomization,Reverberation,Additive noise,Time dropout (temporal masking),Band reject,ClippingInternally, WavAugment useslibsoxand allows interleaving of libsox- and pytorch-based effects.RequirementsLinux or MacOSpytorch>= 1.7.1torchaudio>= 0.7InstallationTo install WavAugment, run the following command:pipinstallwaveaugmentUsageFirst of all, we provide thouroughly documentedexamples, where we demonstrate how a data-augmented dataset interface works. We also provide a Jupyter-basedtutorial(open in colab)that illlustrates how one can apply various useful effects to a piece of speech (recorded over the mic or pre-recorded).TheEffectChainThe central object is the chain of effects,EffectChain, that are applied on atorch.Tensorto produce anothertorch.Tensor.
This chain can have multiple effects composed:importaugmenteffect_chain=augment.EffectChain().pitch(100).rate(16_000)Parameters of the effect coincide with those of libsox (http://sox.sourceforge.net/libsox.html); however, you can also randomize the parameters by providing a pythonCallableand mix them with standard parameters:importnumpyasnprandom_pitch_shift=lambda:np.random.randint(-100,+100)# the pitch will be changed by a shift somewhere between (-100, +100)effect_chain=augment.EffectChain().pitch("-q",random_pitch_shift).rate(16_000)Here, the flag-qmakespitchrun faster at some expense of the quality.
If some parameters are provided by a Callable, this Callable will be invoked every timeEffectChainis applied (eg. to generate random parameters).Applying the chainTo apply a chain of effects on a torch.Tensor, we code the following:output_tensor=augment.EffectChain().pitch(100).rate(16_000).apply(input_tensor,\src_info=src_info,target_info=target_info)WavAugment expectsinput_tensorto have a shape of (channels, length). Asinput_tensordoes not contain important meta-information, such as sampling rate, we need to provide it manually.
This is done by passing two dictionaries,src_info(meta-information about the input format) andtarget_info(our expectated format for the output).At minimum, we need to set the sampling rate for the input tensor:{'rate': 16_000}.Example usageBelow is a small gist of a potential usage:importaugmentimportnumpyasnpx,sr=torchaudio.load(test_wav)# input signal propertiessrc_info={'rate':sr}# output signal propertiestarget_info={'channels':1,'length':0,# not known beforehand'rate':16_000}# write down the chain of effects with their string parameters and call .apply()# effects are specified as a chain of method calls with parameters that can be# strings, numbers, or callables. The latter case is used for generating randomized# transformationsrandom_pitch=lambda:np.random.randint(-400,-200)y=augment.EffectChain().pitch(random_pitch).rate(16_000).apply(x,\src_info=src_info,target_info=target_info)Important notesIt often happens that a command-line invocation of sox would change effect chain. To get a better idea of what sox executes internally, you can launch it with a -V flag, eg by running:sox-Vtests/test.wavout.wavreverb050100we will see something like:sox INFO sox: effects chain: input        16000Hz  1 channels
sox INFO sox: effects chain: reverb       16000Hz  2 channels
sox INFO sox: effects chain: channels     16000Hz  1 channels
sox INFO sox: effects chain: dither       16000Hz  1 channels
sox INFO sox: effects chain: output       16000Hz  1 channelsThis output tells us that thereverbeffect changes the number of channels, which are squashed into 1 channel by thechanneleffect. Sox also addeddithereffect to hide processing artifacts.WavAugment remains explicit and doesn't add effects under the hood.
If you want to emulate a Sox command that decomposes into several effects, we advise to consultsox -Vand apply the effects manually.
Try it out on some files before running a heavy machine-learning job.CitationIf you find WavAugment useful in your research, please consider citing:@article{wavaugment2020,
  title={Data Augmenting Contrastive Learning of Speech Representations in the Time Domain},
  author={Kharitonov, Eugene and Rivi{\`e}re, Morgane and Synnaeve, Gabriel and Wolf, Lior and Mazar{\'e}, Pierre-Emmanuel and Douze, Matthijs and Dupoux, Emmanuel},
  journal={arXiv preprint arXiv:2007.00991},
  year={2020}
}ContributingSee theCONTRIBUTINGfile for how to help out.LicenseWavAugment is MIT licensed, as found in the LICENSE file.