Repository: docker/docker-py

Issue:
port_bindings don't really expose the container
I originally [asked this on SO](http://stackoverflow.com/questions/21112651/how-to-run-basic-web-app-container-in-docker-py/21118935), but since more people confirmed that this seems like unexpected behaviour, I'll report it here as well.

I am trying to mimic running a [containerized web app](https://index.docker.io/u/kermit/hellonode/) exposed at some port

```
sudo docker run -d -p 1338:1337 kermit/hellonode
```

in Python using [docker-py](https://github.com/dotcloud/docker-py). So far I got this code to start the instance:

```
container = c.create_container('kermit/hellonode', name='hello')
c.start(container, port_bindings={1337: ('0.0.0.0', 1338)})
```

But I can't access the container at the public port 1338 (which works normally with the first command) - I'm getting connection refused errors. Does anyone know if I'm missing some option to make the Python call create the functional, accessible container?

Inspecting the container tells me that the port is set up as it should be:

```
$ sudo docker port hello 1337
0.0.0.0:1338
```

I also experimented with the `ports=[1337]` option in the `create_container` call, but it didn't help either.

As [the answer](http://stackoverflow.com/a/21118935/544059) noted, this works if `publish_all_ports` is used, but then you can't control the port number.

```
container = c.create_container('kermit/hellonode', name='hello', ports=[1337])
c.start(container, publish_all_ports=True)
info = c.inspect_container(container)
host_port = info['NetworkSettings']['Ports']['1337'][0]['HostPort']
```



Discussion:
Hmm, what version of `docker-py` are you using? Is it latest `master`?

I think you're not using the correct combination of parameters in the `create_container()` and `start()` calls. You had both right, but not at the same time. As you can see I'm not able to reproduce:

```
30> c.containers()
30: []

31> container = c.create_container('stackbrew/ubuntu:12.04',
        command=['sleep', '15'], ports=[1338])

32> c.start(container, port_bindings={1337: ('0.0.0.0', 1338)})

33> c.inspect_container(container)
33:
{u'Args': [u'15'],
 u'Config': {u'AttachStderr': True,
  u'AttachStdin': False,
  u'AttachStdout': True,
  u'Cmd': [u'sleep', u'15'],
  u'CpuShares': 0,
  u'Dns': None,
  u'Domainname': u'',
  u'Entrypoint': None,
  u'Env': [u'HOME=/',
   u'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'],
  u'ExposedPorts': {u'1338': {}},
  u'Hostname': u'a273f19ab1ba',
  u'Image': u'stackbrew/ubuntu:12.04',
  u'Memory': 0,
  u'MemorySwap': 0,
  u'NetworkDisabled': False,
  u'OpenStdin': False,
  u'PortSpecs': None,
  u'StdinOnce': False,
  u'Tty': False,
  u'User': u'',
  u'Volumes': None,
  u'VolumesFrom': u'',
  u'WorkingDir': u''},
 u'Created': u'2014-01-14T18:07:59.031292786Z',
 u'Driver': u'aufs',
 u'HostConfig': {u'Binds': None,
  u'ContainerIDFile': u'',
  u'Links': None,
  u'LxcConf': None,
  u'PortBindings': {u'1337/tcp': [{u'HostIp': u'0.0.0.0',
     u'HostPort': u'1338'}],
   u'1338': None},
  u'Privileged': False,
  u'PublishAllPorts': False},
 u'HostnamePath': u'/var/lib/docker/containers/a273f19ab1babb363fe8b209a55c64c6cf9430eb5fc65f42eb1a0668529ba38c/hostname',
 u'HostsPath': u'/var/lib/docker/containers/a273f19ab1babb363fe8b209a55c64c6cf9430eb5fc65f42eb1a0668529ba38c/hosts',
 u'ID': u'a273f19ab1babb363fe8b209a55c64c6cf9430eb5fc65f42eb1a0668529ba38c',
 u'Image': u'3aa646e4f1d2bd718c20174a38a87ac33a223ccd70fde655185197541ab6fcab',
 u'Name': u'/prickly_thompson4',
 u'NetworkSettings': {u'Bridge': u'docker0',
  u'Gateway': u'172.17.42.1',
  u'IPAddress': u'172.17.0.62',
  u'IPPrefixLen': 16,
  u'PortMapping': None,
  u'Ports': {u'1337/tcp': [{u'HostIp': u'0.0.0.0', u'HostPort': u'1338'}],
   u'1338': None}},
 u'Path': u'sleep',
 u'ResolvConfPath': u'/etc/resolv.conf',
 u'State': {u'ExitCode': 0,
  u'FinishedAt': u'0001-01-01T00:00:00Z',
  u'Ghost': False,
  u'Pid': 17661,
  u'Running': True,
  u'StartedAt': u'2014-01-14T18:08:02.908042711Z'},
 u'Volumes': {},
 u'VolumesRW': {}}

34> c.containers()
34:
[{u'Command': u'sleep 15',
  u'Created': 1389722879,
  u'Id': u'a273f19ab1babb363fe8b209a55c64c6cf9430eb5fc65f42eb1a0668529ba38c',
  u'Image': u'stackbrew/ubuntu:12.04',
  u'Names': [u'/prickly_thompson4'],
  u'Ports': [{u'IP': u'0.0.0.0',
    u'PrivatePort': 1337,
    u'PublicPort': 1338,
    u'Type': u'tcp'},
   {u'IP': u'', u'PrivatePort': 0, u'PublicPort': 1338, u'Type': u'tcp'}],
  u'SizeRootFs': 0,
  u'SizeRw': 0,
  u'Status': u'Up 8 seconds'}]

35>
```

I'm using docker-py 0.2.3 (installed using pip) and Python 3.3.3.

My problem is not that the container won't start (everything appears normal), but that I can't connect to it through the public port (and this does work when I start the container using the docker cli).

``` python
In [7]: container = c.create_container('kermit/hellonode', name='hello', ports=[1337])

In [8]: c.start(container, port_bindings={1337: ('0.0.0.0', 1338)})

In [9]: c.inspect_container(container)
Out[9]:
{'Args': ['-c', 'node /hello.js'],
 'Config': {'AttachStderr': True,
  'AttachStdin': False,
  'AttachStdout': True,
  'Cmd': ['/bin/sh', '-c', 'node /hello.js'],
  'CpuShares': 0,
  'Dns': None,
  'Domainname': '',
  'Entrypoint': None,
  'Env': ['HOME=/',
   'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'],
  'ExposedPorts': {'1337': {}},
  'Hostname': '2c4f953e10fc',
  'Image': 'kermit/hellonode',
  'Memory': 0,
  'MemorySwap': 0,
  'NetworkDisabled': False,
  'OpenStdin': False,
  'PortSpecs': None,
  'StdinOnce': False,
  'Tty': False,
  'User': '',
  'Volumes': None,
  'VolumesFrom': '',
  'WorkingDir': ''},
 'Created': '2014-01-14T20:44:36.33498228Z',
 'Driver': 'aufs',
 'HostConfig': {'Binds': None,
  'ContainerIDFile': '',
  'Links': None,
  'LxcConf': None,
  'PortBindings': {'1337': None,
   '1337/tcp': [{'HostIp': '0.0.0.0', 'HostPort': '1338'}]},
  'Privileged': False,
  'PublishAllPorts': False},
 'HostnamePath': '/var/lib/docker/containers/2c4f953e10fcf2ca1ba0c6f100be1191de5f9d95b672b9cf5d4be9b66b0dbdde/hostname',
 'HostsPath': '/var/lib/docker/containers/2c4f953e10fcf2ca1ba0c6f100be1191de5f9d95b672b9cf5d4be9b66b0dbdde/hosts',
 'ID': '2c4f953e10fcf2ca1ba0c6f100be1191de5f9d95b672b9cf5d4be9b66b0dbdde',
 'Image': 'fbccec340a0ee0f8248a69a23f6dee612464fec24ceb262f209ca413fa26ea5a',
 'Name': '/hello',
 'NetworkSettings': {'Bridge': 'docker0',
  'Gateway': '172.17.42.1',
  'IPAddress': '172.17.0.25',
  'IPPrefixLen': 16,
  'PortMapping': None,
  'Ports': {'1337': None,
   '1337/tcp': [{'HostIp': '0.0.0.0', 'HostPort': '1338'}]}},
 'Path': '/bin/sh',
 'ResolvConfPath': '/etc/resolv.conf',
 'State': {'ExitCode': 0,
  'FinishedAt': '0001-01-01T00:00:00Z',
  'Ghost': False,
  'Pid': 16423,
  'Running': True,
  'StartedAt': '2014-01-14T20:44:38.984802489Z'},
 'Volumes': {},
 'VolumesRW': {}}

In [10]: ! curl blimpyard.cloudfleet.io:1338
curl: (7) Failed connect to blimpyard.cloudfleet.io:1338; Connection refused
```

These steps should be reproducible (aside from the url passed to curl) - I pushed the container to the docker index. The web app is the _helloworld_ example from [node's homepage](http://nodejs.org/).

``` javascript
var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
```

Ok, I found what the problem was. When creating the container, if you don't define the protocol type (TCP or UDP), the `HostConfig` will contain an invalid port mapping for the same port that was requested on the `start()` call:

```
  u'Ports': {u'8000': None,
   u'8000/tcp': [{u'HostIp': u'0.0.0.0', u'HostPort': u'8082'}]}},
```

This seems to confuse Docker and it doesn't create the `iptables` NAT rule. If you do the following, it works:

```
43> container = c.create_container('stackbrew/ubuntu:12.04',
    command=['python', '-m', 'SimpleHTTPServer'], ports=[(8000, 'tcp')])
43> c.start(container, port_bindings={8000: ('0.0.0.0', 8082)})
44> c.inspect_container(container)['HostConfig']
44:
{u'Binds': None,
 u'ContainerIDFile': u'',
 u'Links': None,
 u'LxcConf': None,
 u'PortBindings': {u'8000/tcp': [{u'HostIp': u'0.0.0.0',
    u'HostPort': u'8082'}]},
 u'Privileged': False,
 u'PublishAllPorts': False}

45> c.inspect_container(container)['NetworkSettings']
45:
{u'Bridge': u'docker0',
 u'Gateway': u'172.17.42.1',
 u'IPAddress': u'172.17.0.99',
 u'IPPrefixLen': 16,
 u'PortMapping': None,
 u'Ports': {u'8000/tcp': [{u'HostIp': u'0.0.0.0', u'HostPort': u'8082'}]}}
```

Checks out:

```
$ sudo iptables -t nat -L -v -n | grep 8000
    0     0 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8082 to:172.17.0.99:8000
$ docker ps
CONTAINER ID        IMAGE                                                           COMMAND                CREATED             STATUS              PORTS                                                                                              NAMES
159381063215        stackbrew/ubuntu:12.04                                          python -m SimpleHTTP   26 seconds ago      Up 6 seconds        0.0.0.0:8082->8000/tcp                                                                             tender_archimedes
$ echo "HEAD / HTTP/1.0" | nc localhost 8082
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/2.7.3
Date: Wed, 15 Jan 2014 00:28:16 GMT
Content-type: text/html; charset=ANSI_X3.4-1968
Content-Length: 846
```

@shin- If TCP is the default for port redirects, I think we should modify `_container_config()` to set `proto` to `tcp` if it's not provided, defined or is empty, to avoid this problem. What do you think?

Is it because your server is bound to 127.0.0.1? Have you tried binding to 0.0.0.0 instead? I'm not sure this is a good solution but it might be _a_ solution...

Edit: never mind!

@mpetazzoni thanks, that worked! Yes, I also think it should either be set to TCP automatically to avoid these mishaps or at least documented properly as [the docs currently state](https://github.com/dotcloud/docker-py#port-bindings) that TCP is the default.

> If you wish to use UDP instead of TCP (default), you can declare it like such:
>    c.create_container('busybox', 'ls', ports=[(1111, 'udp'), 2222])

@joshuaconner good spotting, but it's a typing mistake because I just pasted the code from the node website, not from my actual script - I do set the web app to listen to 0.0.0.0. Anyway, it worked for the same container image with the docker cli.

If I remember correctly when the new port bindings and volumes API was being implemented this was supposed to be the case, meaning to set /tcp if nothing was specified by the user, so this is most likely a bug.

@denibertovic In Docker itself you mean? I really find it strange that Docker reports a port number without either `/udp` or `/tcp` at the end.

@mpetazzoni no I mean when we were discussing how to make the docker-py methods more pythonic, we agreed to make 'tcp' the default.

Also, I think I've stumbled upon this myself. I just forgot about it because at the time i prepended 'tcp' manually in my code, whereas docker-py should do it. Upon inspecting the container I saw that the port was without the proto part.

This might be a bug in the Docker API as well, because the docker API should raise an error if no proto is specified right?

Agreed that it's a bug, `tcp` should be the default. And yeah, if "no protocol" is invalid, Docker API shouldn't accept it.

So, what's the final word? I agree that the Docker API shouldn't accept it, but to prevent people from falling into the trap/bug while using `docker-py` we could default to `tcp` in `create_container()` if the protocol is not specified, what do you think?


Resolving patch:
diff --git a/docker/client.py b/docker/client.py
--- a/docker/client.py
+++ b/docker/client.py
@@ -134,15 +134,12 @@ def _container_config(self, image, command, hostname=None, user=None,
             exposed_ports = {}
             for port_definition in ports:
                 port = port_definition
-                proto = None
+                proto = 'tcp'
                 if isinstance(port_definition, tuple):
                     if len(port_definition) == 2:
                         proto = port_definition[1]
                     port = port_definition[0]
-                exposed_ports['{0}{1}'.format(
-                    port,
-                    '/' + proto if proto else ''
-                )] = {}
+                exposed_ports['{0}/{1}'.format(port, proto)] = {}
             ports = exposed_ports
 
         if volumes and isinstance(volumes, list):


Test patch:
diff --git a/tests/test.py b/tests/test.py
--- a/tests/test.py
+++ b/tests/test.py
@@ -223,9 +223,9 @@ def test_create_container_with_ports(self):
                             {"Tty": false, "Image": "busybox",
                              "Cmd": ["ls"], "AttachStdin": false,
                              "Memory": 0, "ExposedPorts": {
-                                "1111": {},
+                                "1111/tcp": {},
                                 "2222/udp": {},
-                                "3333": {}
+                                "3333/tcp": {}
                              },
                              "AttachStderr": true,
                              "AttachStdout": true, "OpenStdin": false,