Fix possible memory bug

In case the vector implementation shrinks the underlying
array.