Make remote_ip detection properly handle private IPv6 addresses

Fixes #12638.