On empty whitelist_ip we should not use it.