Remove hole in the middle

There is additional cleanup that will come with the `DirPath` merge but at least the hole-in-the-middle is gone now.
Can you share this bit shifting with `unsafe`? (so there is no duplicate logic)