Why not just extract the data and drop the reference to the buffer?