Handle remote address parsing better

Change-Id: I84dec22ab92a1f2b440948a0e029bc51aad75fdb