Make remote_ip detection properly handle private IPv6 addresses

Fixes #12638.

I didn't see any relevant tests to change.