Updated IP Whitelisting to be port-conscious as RemoteAddr has no specific format, but we need to look for IPv4 and IPv6 addresses