Sweep unused buffers