Split the bucket vector and the summary vector to reduce the number of double conversions