Better latency buffer handling