Instead of making it unsafe, could these properties be asserted at runtime? Resizing a hash map will never be a fast operation, so adding a few minorly expensive assertions shouldn't hurt the performance that much.

This function is called fairly frequently from the below code, and it's a shame to introduce `unsafe` code when the invariants can be asserted at runtime.