Move out remote address parsing and fix bugs