not really in-place but "returns the actual index vector memory, a newly allocated vector otherwise"