Make sure the buffer is free'd before receives are resumed to fix possible race