Fix client whitelisting for log input

When receiving a new connection, the code was checking the local ip address to see if it was allowed,
This fixes it to look at the remote ip address.