Repository: mir-evaluation/mir_eval
Issue: continuous pitch sonification
GitHub labels: enhancement

As per [this discussion](https://github.com/marl/jams/pull/91), we could pull in the code from @justinsalamon 's [melosynth](https://github.com/justinsalamon/melosynth) package.


tagging @justinsalamon @craffel 

I hacked out a quick [gist](https://gist.github.com/935c7c7e61824cfe2c5708eeed10dc83) for pitch contour sonification, based on how I understand the code in melosynth.  This version is much simpler (and, I suspect, much faster) since it uses a scipy interpolator to do the instantaneous frequency estimation.  It also works like the other sonifiers, allowing the user to supply a waveform function (sin, square, etc) directly, rather than modeling the harmonics explicitly.

The results sound good to me, but a sanity check from the experts would be appreciated before I polish and PR.


Can you post `train05REF.jams` somewhere so that I can try it out?  The `np.cumsum` is throwing me, what is that doing?  It seems like it would only do the right thing when `f_est` was constant.


@craffel https://github.com/marl/jams-data/blob/master/datasets/MIREX05/train05REF.jams


> The results sound good to me, but a sanity check from the experts would be appreciated before I polish and PR.

Sounds fine to me


> The np.cumsum is throwing me, what is that doing? It seems like it would only do the right thing when f_est was constant.

The analogous part is [here](https://github.com/justinsalamon/melosynth/blob/master/melosynth.py#L330-L331) -- phase propagation between samples.  I just didn't bother computing the phase separately, since it's not needed.

I played the output of this side-by-side with melosynth, and they agree to the extent that's necessary; tapering behavior is a little different, but the pitch and timing are essentially identical.


> The analogous part is here -- phase propagation between samples. I just didn't bother computing the phase separately, since it's not needed.

Ok, I am seeing it now.

> I played the output of this side-by-side with melosynth, and they agree to the extent that's necessary; tapering behavior is a little different, but the pitch and timing are essentially identical.

I would have thought the clicking would have been worse the way you're doing it, but I'm not hearing it even at higher sampling rate, so I am ok with doing it your way.  Anyways, LGTM, feel free to PR, thanks.


> I would have thought the clicking would have been worse the way you're doing it, but I'm not hearing it even at higher sampling rate, so I am ok with doing it your way.

The trick here is clipping the unvoiced frequency measurements to 0, and then interpolating; this ensures at least a ramp (for linear) or smoother transition (quadratic, cubic) whenever entering or leaving voiced regions.  It's a little sharper than @justinsalamon's version since I don't explicitly force a minimum width of the ramp, so you do see some spread in the spectrogram at the low frequencies in my version.  I'm fine with this as well.  PR incoming.


> The trick here is clipping the unvoiced frequency measurements to 0, and then interpolating; this ensures at least a ramp (for linear) or smoother transition (quadratic, cubic) whenever entering or leaving voiced regions.

Right, my worry was that because you are ramping smoothly in frequency rather than amplitude you might get some weird effects, but it seems ok.