Get rid of the separate array for merged buffers