Get rid of all interleaving code, just try to estimate the time based on PTS/DTS values.

It's less accurate, but at least it works with all versions of ffmpeg/libav.