Repository: ipython/ipython

Issue:
counterintuitive behavior of ipengine when controller is listening on all interfaces.
When I start ipcontroller as follows:

```
ipcontroller --ip='172.16.1.140'
```

(where 173.16.1.140 is the public ip of the machine)

and then on remote hosts start ipengine as follows:

```
ipengine --location=172.16.1.140
```

everything works fine.

However if I instead start ipcontroller with:

```
ipcontroller --ip='*' 
```

it starts up fine, but then trying to start a controller on a remote host with:

```
ipengine --location=172.16.1.140 --log-level=DEBUG
```

gives 

```
2014-04-01 00:26:55.408 [IPEngineApp] Config changed:
2014-04-01 00:26:55.409 [IPEngineApp] {'EngineFactory': {'location': u'172.16.1.140'}, 'IPEngineApp': {'log_level': u'DEBUG'}}
2014-04-01 00:26:55.488 [IPEngineApp] Using existing profile dir: u'/glusterfs/users/PORTERJAMESJ/.config/ipython/profile_default'
2014-04-01 00:26:55.488 [IPEngineApp] Searching path [u'/glusterfs/users/PORTERJAMESJ', u'/glusterfs/users/PORTERJAMESJ/.config/ipython/profile_default'] for config files
2014-04-01 00:26:55.488 [IPEngineApp] Attempting to load config file: ipython_config.py
2014-04-01 00:26:55.529 [IPEngineApp] Loaded config file: /glusterfs/users/PORTERJAMESJ/.config/ipython/profile_default/ipython_config.py
2014-04-01 00:26:55.529 [IPEngineApp] Attempting to load config file: ipengine_config.py
2014-04-01 00:26:55.605 [IPEngineApp] Config file not found, skipping: ipcontroller_config.py
2014-04-01 00:26:55.605 [IPEngineApp] Attempting to load config file: ipengine_config.py
2014-04-01 00:26:55.745 [IPEngineApp] Config file not found, skipping: ipengine_config.py
2014-04-01 00:26:55.745 [IPEngineApp] Attempting to load config file: ipengine_config.py
2014-04-01 00:26:55.769 [IPEngineApp] Config file not found, skipping: ipcluster_config.py
2014-04-01 00:26:55.805 [IPEngineApp] Loading url_file u'/glusterfs/users/PORTERJAMESJ/.config/ipython/profile_default/security/ipcontroller-engine.json'
2014-04-01 00:26:55.815 [IPEngineApp] Config changed:
2014-04-01 00:26:55.815 [IPEngineApp] {'RegistrationFactory': {}, 'EngineFactory': {'url': u'tcp://127.0.0.1:59799', 'location': u'172.16.1.140', 'sshserver': u''}, 'Session': {'packer': u'json', 'unpacker': u'json', 'signature_scheme': u'hmac-sha256', 'key': '7cf13c1c-00b0-4570-ac76-4899c4560783'}, 'IPEngineApp': {'log_level': u'DEBUG'}, 'ProfileDir': {}}
2014-04-01 00:26:56.074 [IPEngineApp] Registering with controller at tcp://127.0.0.1:59799
2014-04-01 00:27:01.078 [IPEngineApp] CRITICAL | Registration timed out after 5.0 seconds
```

note that the engine tries to register on localhost instead of the `location` specified in the url file, which is correct, and for reference looks like the following:

```
{
  "control": 44663,
  "task": 52709,
  "hb_ping": 33695,
  "mux": 58852,
  "pack": "json",
  "hb_pong": 50757,
  "ssh": "",
  "key": "7cf13c1c-00b0-4570-ac76-4899c4560783",
  "registration": 59799,
  "interface": "tcp://*",
  "iopub": 48465,
  "signature_scheme": "hmac-sha256",
  "unpack": "json",
  "location": "172.16.1.140"
}
```

As far as I can tell this is due to the way [`disambiguate_ip_address`](https://github.com/ipython/ipython/blob/master/IPython/parallel/util.py#L188-L197) works. It's called [here](https://github.com/ipython/ipython/blob/master/IPython/parallel/apps/ipengineapp.py#L232) when the engine is launching. This behavior is confusing to me; why shoudn't the engine use the location specified in the url_file when the interface is `*`?



Discussion:
<del>
--location is a shortcut for bypassing IPython's guess about the IP of the machine on which it is running. When you pass ipengine --location=172.16.1.140 you are telling ipengine that *ipengine* is run on the machine 172.16.1.140. ipengine then compares this with the `location` field in the connection file (the controller's IP, and set by ipcontroller --location if you want), and determines that the engine and controller are running on the same machine. Finally, since the controller is listening on all interfaces (*), the engine should use the loopback interface.
</del>

<del>
Basically, you told ipengine explicitly that the engine is on the same machine as the controller.
</del>

oh, I see. The documentation seems to contradict this. `ipengine -h` gives:

```
--location=<Unicode> (EngineFactory.location)
    Default: u''
    The location (an IP address) of the controller.  This is used for
    disambiguating URLs, to determine whether loopback should be used to connect
    or the public address.
```

Also this issue persists regardless of how I set `--location`. Leaving it off entirely or setting it to the actual ip address of the machine I'm starting the engine on seems to give the same behavior. Regardless of how I start the engine, it tries to connect to localhost instead of the `location` specified in the url_file. I'll look into it more tomorrow.

Sorry for the misinformation, `location` is the controller's IP as described in the help string.  Specifying this to `ipengine` is redundant unless the `location` key in the connection file is incorrect for some reason.

What is the output of

```
from IPython.utils import localinterfaces
print(localinterfaces.public_ips())
```

on your engine?

I'm on 1.x so I don't have `public_ips` but `PUBLIC_IPS` is an empty list. Do engines also need to know their own IP?


Resolving patch:
diff --git a/IPython/parallel/util.py b/IPython/parallel/util.py
--- a/IPython/parallel/util.py
+++ b/IPython/parallel/util.py
@@ -1,27 +1,15 @@
-"""Some generic utilities for dealing with classes, urls, and serialization.
+"""Some generic utilities for dealing with classes, urls, and serialization."""
 
-Authors:
+# Copyright (c) IPython Development Team.
+# Distributed under the terms of the Modified BSD License.
 
-* Min RK
-"""
-#-----------------------------------------------------------------------------
-#  Copyright (C) 2010-2011  The IPython Development Team
-#
-#  Distributed under the terms of the BSD License.  The full license is in
-#  the file COPYING, distributed as part of this software.
-#-----------------------------------------------------------------------------
-
-#-----------------------------------------------------------------------------
-# Imports
-#-----------------------------------------------------------------------------
-
-# Standard library imports.
 import logging
 import os
 import re
 import stat
 import socket
 import sys
+import warnings
 from signal import signal, SIGINT, SIGABRT, SIGTERM
 try:
     from signal import SIGKILL
@@ -36,13 +24,11 @@
     cPickle = None
     import pickle
 
-# System library imports
 import zmq
 from zmq.log import handlers
 
 from IPython.external.decorator import decorator
 
-# IPython imports
 from IPython.config.application import Application
 from IPython.utils.localinterfaces import localhost, is_public_ip, public_ips
 from IPython.utils.py3compat import string_types, iteritems, itervalues
@@ -184,18 +170,42 @@ def split_url(url):
     assert len(lis) == 2, 'Invalid url: %r'%url
     addr,s_port = lis
     return proto,addr,s_port
-    
+
+
 def disambiguate_ip_address(ip, location=None):
-    """turn multi-ip interfaces '0.0.0.0' and '*' into connectable
-    ones, based on the location (default interpretation of location is localhost)."""
-    if ip in ('0.0.0.0', '*'):
-        if location is None or is_public_ip(location) or not public_ips():
-            # If location is unspecified or cannot be determined, assume local
+    """turn multi-ip interfaces '0.0.0.0' and '*' into a connectable address
+    
+    Explicit IP addresses are returned unmodified.
+    
+    Parameters
+    ----------
+    
+    ip : IP address
+        An IP address, or the special values 0.0.0.0, or *
+    location: IP address, optional
+        A public IP of the target machine.
+        If location is an IP of the current machine,
+        localhost will be returned,
+        otherwise location will be returned.
+    """
+    if ip in {'0.0.0.0', '*'}:
+        if not location:
+            # unspecified location, localhost is the only choice
+            ip = localhost()
+        elif is_public_ip(location):
+            # location is a public IP on this machine, use localhost
             ip = localhost()
-        elif location:
-            return location
+        elif not public_ips():
+            # this machine's public IPs cannot be determined,
+            # assume `location` is not this machine
+            warnings.warn("IPython could not determine public IPs", RuntimeWarning)
+            ip = location
+        else:
+            # location is not this machine, do not use loopback
+            ip = location
     return ip
 
+
 def disambiguate_url(url, location=None):
     """turn multi-ip interfaces '0.0.0.0' and '*' into connectable
     ones, based on the location (default interpretation is localhost).


Test patch: