WIP: Add source address to PoolManager

This is #376 after a rebase. Shortly I'll address the review comments there and get this into a merge ready state.
I don't think we need these here, since they'll ultimately hit the equivalent in connection.py, no?