Reduce the memory copy while building the hashmap