Revert "Add a minimum buffer zize to the audio buffer"

This reverts commit f34aabe41dde20e6a00e8b2066e8a8b9abc80e1f.