Swap buffers only when idle