Reduce HashMap allocations

We previously allocated 3x for every HashMap creation and resize. This patch reduces it to 1x.