Only add the vector if we need to