Package: torchcrepe
torchcrepePytorch implementation of the CREPE [1] pitch tracker. The original Tensorflow
implementation can be foundhere. The
provided model weights were obtained by converting the "tiny" and "full" models
usingMMdnn, an open-source model
management framework.InstallationPerform the system-dependent PyTorch install using the instructions foundhere.pip install torchcrepeUsageComputing pitch and periodicity from audioimporttorchcrepe# Load audioaudio,sr=torchcrepe.load.audio(...)# Here we'll use a 5 millisecond hop lengthhop_length=int(sr/200.)# Provide a sensible frequency range for your domain (upper limit is 2006 Hz)# This would be a reasonable range for speechfmin=50fmax=550# Select a model capacity--one of "tiny" or "full"model='tiny'# Choose a device to use for inferencedevice='cuda:0'# Pick a batch size that doesn't cause memory errors on your gpubatch_size=2048# Compute pitch using first gpupitch=torchcrepe.predict(audio,sr,hop_length,fmin,fmax,model,batch_size=batch_size,device=device)A periodicity metric similar to the Crepe confidence score can also be
extracted by passingreturn_periodicity=Truetotorchcrepe.predict.DecodingBy default,torchcrepeuses Viterbi decoding on the softmax of the network
output. This is different than the original implementation, which uses a
weighted average near the argmax of binary cross-entropy probabilities.
The argmax operation can cause double/half frequency errors. These can be
removed by penalizing large pitch jumps via Viterbi decoding. Thedecodesubmodule provides some options for decoding.# Decode using viterbi decoding (default)torchcrepe.predict(...,decoder=torchcrepe.decode.viterbi)# Decode using weighted argmax (as in the original implementation)torchcrepe.predict(...,decoder=torchcrepe.decode.weighted_argmax)# Decode using argmaxtorchcrepe.predict(...,decoder=torchcrepe.decode.argmax)Filtering and thresholdingWhen periodicity is low, the pitch is less reliable. For some problems, it
makes sense to mask these less reliable pitch values. However, the periodicity
can be noisy and the pitch has quantization artifacts.torchcrepeprovides
submodulesfilterandthresholdfor this purpose. The filter and threshold
parameters should be tuned to your data. For clean speech, a 10-20 millisecond
window with a threshold of 0.21 has worked.# We'll use a 15 millisecond window assuming a hop length of 5 millisecondswin_length=3# Median filter noisy confidence valueperiodicity=torchcrepe.filter.median(periodicity,win_length)# Remove inharmonic regionspitch=torchcrepe.threshold.At(.21)(pitch,periodicity)# Optionally smooth pitch to remove quantization artifactspitch=torchcrepe.filter.mean(pitch,win_length)For more fine-grained control over pitch thresholding, seetorchcrepe.threshold.Hysteresis. This is especially useful for removing
spurious voiced regions caused by noise in the periodicity values, but
has more parameters and may require more manual tuning to your data.CREPE was not trained on silent audio. Therefore, it sometimes assigns high
confidence to pitch bins in silent regions. You can usetorchcrepe.threshold.Silenceto manually set the periodicity in silent
regions to zero.periodicity=torchcrepe.threshold.Silence(-60.)(periodicity,audio,sr,hop_length)Computing the CREPE model output activationsbatch=next(torchcrepe.preprocess(audio,sr,hop_length))probabilities=torchcrepe.infer(batch)Computing the CREPE embedding spaceAs in Differentiable Digital Signal Processing [2], this uses the output of the
fifth max-pooling layer as a pretrained pitch embeddingembeddings=torchcrepe.embed(audio,sr,hop_length)Computing from filestorchcrepedefines the following functions convenient for predicting
directly from audio files on disk. Each of these functions also takes
adeviceargument that can be used for device placement (e.g.,device='cuda:0').torchcrepe.predict_from_file(audio_file,...)torchcrepe.predict_from_file_to_file(audio_file,output_pitch_file,output_periodicity_file,...)torchcrepe.predict_from_files_to_files(audio_files,output_pitch_files,output_periodicity_files,...)torchcrepe.embed_from_file(audio_file,...)torchcrepe.embed_from_file_to_file(audio_file,output_file,...)torchcrepe.embed_from_files_to_files(audio_files,output_files,...)Command-line interfaceusage:python-mtorchcrepe[-h]--audio_filesAUDIO_FILES[AUDIO_FILES...]--output_filesOUTPUT_FILES[OUTPUT_FILES...][--hop_lengthHOP_LENGTH][--output_periodicity_filesOUTPUT_PERIODICITY_FILES[OUTPUT_PERIODICITY_FILES...]][--embed][--fminFMIN][--fmaxFMAX][--modelMODEL][--decoderDECODER][--gpuGPU][--no_pad]optionalarguments:-h,--helpshowthishelpmessageandexit--audio_filesAUDIO_FILES[AUDIO_FILES...]Theaudiofiletoprocess--output_filesOUTPUT_FILES[OUTPUT_FILES...]Thefiletosavepitchorembedding--hop_lengthHOP_LENGTHThehoplengthoftheanalysiswindow--output_periodicity_filesOUTPUT_PERIODICITY_FILES[OUTPUT_PERIODICITY_FILES...]Thefiletosaveperiodicity--embedPerformsembeddinginsteadofpitchprediction--fminFMINTheminimumfrequencyallowed--fmaxFMAXThemaximumfrequencyallowed--modelMODELThemodelcapacity.Oneof"tiny"or"full"--decoderDECODERThedecodertouse.Oneof"argmax","viterbi",or"weighted_argmax"--gpuGPUThegputoperforminferenceon--no_padWhethertopadtheaudioTestsThe module tests can be run as follows.pipinstallpytest
pytestReferences[1] J. W. Kim, J. Salamon, P. Li, and J. P. Bello, “Crepe: A
Convolutional Representation for Pitch Estimation,” in 2018 IEEE
International Conference on Acoustics, Speech and Signal
Processing (ICASSP).[2] J. H. Engel, L. Hantrakul, C. Gu, and A. Roberts,
“DDSP: Differentiable Digital Signal Processing,” in
2020 International Conference on Learning
Representations (ICLR).