Finally fixed the audio!

Problem was that many buffers were going undeleted.
Turns out you cannot just rely on the source to immediately
mark all buffers as processed -- you have to keep track
of which buffers are in the source yourself, and wait for
them all to be removed.

TODO: This code could still leak buffers if a buffer is created but
not added to the set because the thread is killed in between.