That makes sense. And I just found that it failed the test "Should not copy data".  SparseVector from VectorBuilder has this problem of size inconsistency. It uses a dynamic growing approach to resize the index/data array and fills the unused slot zeros. So I think there is no way to completely avoid new allocation.