[Server] Fix IPv6 server addresses with port