That's a legitimate concern, and one I think I have an unsatisfactory answer for.

I didn't take any perf measurements between the safe and unsafe variants. But either way, I can't see how "safe" code can possibly be just as fast as "unsafe" code. Hashtables rely on the invariant that part of the array is uninitialized, and you can see this unsafety show up in other low level data structures such as in libstd/sync/deque.rs. This invariant can't really be enforced statically, and without it you bloat the code a lot with things that are "impossible".

The fact that rust lets me get this close to the metal and still provide a safe interface with a nice type system on top is one of the reasons I'm falling in love with it.