WIP: Add source address to PoolManager

This is #376 after a rebase. Shortly I'll address the review comments there and get this into a merge ready state.
Oh I see, we're popping it to avoid repeating warnings. Hm.

Not sure that's worth it? Lots of litter for it.