Another good thing for these temp buffers is to have a boost::scoped_array<>, and a second size_t with the allocated size.  The scoped_array auto-deletes itself like a vector so you can't leak memory, but unlike a vector, it does not zero out POD types upon allocation.

If you don't immediately get what I'm suggesting, then just keep it as a vector, let's get it committed with correct code, then we can separately address performance in a second pass over this code (and I can explain in more detail how to us the scoped_array).