Fix a potential race condition when refreshing the buffers table