WIP: fix persistent port allocation

This piggybacks on #6555 with some fixups according to @lk4d4's comments. More to come, but please feel free to review at any point.

/cc @crosbymichael 
I couldn't find where this struct is used. Leftovers?