xfer: Fix segfault in child process.

This was broken before the previous change, since remote_address was
just a zero'd out struct.