[ISSUE-95] Send replies from the right source IP address

Fixes #95

I didn't test this enough, but lets see what you have to say about it :)
I tried to keep it as simple as possible.

This has to be tested with IPv6. I guess that IPv6 is not supported yet (shouldn't be too hard to add IPv6 support though), but first lets make sure it doesn't break it.
I think it will be better to add a comment explaining why this is "empty". At least add reference to this pull request.