WIP: fix persistent port allocation

This piggybacks on #6555 with some fixups according to @lk4d4's comments. More to come, but please feel free to review at any point.

/cc @crosbymichael 
I think this comment doesn't make sense anymore.