Don't create a vector on the stack for every merge we do
Write directly into scratch vector at offset rather than pushing to it like a stack