Repository: deis/deis

Issue:
nginx configuration incorrect
I think the current nginx configuration on the router is incorrect.

It takes everything that is advertised in /etc/services/<app_name>/host:port and sticks them under the same subdomain in nginx. 

however when I have an application that has 3 processes for instance (eg web, worker, and clock) it should _only_ use the web instances and stick them in nginx. Everything else should really be ignored. So might be worth considering not having them added in there in the first place.

currently the bug on #1251 might be due to this (at least on my case) as the 'clock' process doesn't need any external interface (neither does the 'worker' process).

I guess it applies mainly for buildpacks where we can expose multiple services through the same build.

This is the line of code in question: https://github.com/deis/deis/blob/master/router/templates/nginx.conf#L62-L67



Discussion:
Wondering what the solution is here - I don't know if we can reliably always choose the "web" process, as I think we have some flexibility in process type naming. How would we denote if a process type should be exposed in the router or not?

From https://devcenter.heroku.com/articles/dynos#features:

> Routing: The routers tracks the location of all running dynos with the web process type (web dynos) and routes HTTP traffic to them accordingly.

I think it'd be a reasonable assumption to say that only `web` or `cmd` processes can go through the router for now. If/when TCP support is enabled for the router, then we can start looking at other process types to expose from the router. Does that seem reasonable?

@bacongobbler Do cmd types (Dockerfiles) have an implicit web type, or is that a special case? That sounds like a reasonable approach to me, but I'd like @gabrtv to weigh in as well.

> @bacongobbler Do cmd types (Dockerfiles) have an implicit web type, or is that a special case?

Yeah. In terms of process types, cmd == web :)

> I think it'd be a reasonable assumption to say that only web or cmd processes can go through the router for now. If/when TCP support is enabled for the router, then we can start looking at other process types to expose from the router. Does that seem reasonable?

Yes!

@bacongobbler If you get a few minutes, can you whip up a PR for this so @developerinlondon can see if it's a resolution? 

@ddollar saw the same issue on a web process type, though, so don't think this solves all occurrences.


Resolving patch:
diff --git a/controller/scheduler/coreos.py b/controller/scheduler/coreos.py
--- a/controller/scheduler/coreos.py
+++ b/controller/scheduler/coreos.py
@@ -56,7 +56,12 @@ def create(self, name, image, command='', template=None):
         env = self.env.copy()
         self._create_container(name, image, command, template or CONTAINER_TEMPLATE, env)
         self._create_log(name, image, command, LOG_TEMPLATE, env)
-        self._create_announcer(name, image, command, ANNOUNCE_TEMPLATE, env)
+
+        # only announce web and cmd processes
+        if command.lower() in ['start web', '']:
+            self._create_announcer(name, image, command, ANNOUNCE_TEMPLATE, env)
+        else:
+            print "-- skipping announcer for {} - cmd type is {}".format(name,command)
 
     def _create_container(self, name, image, command, template, env):
         l = locals().copy()
@@ -202,6 +207,7 @@ def attach(self, name):
 TimeoutStartSec=20m
 """
 
+# TODO revisit the "not getting a port" issue after we upgrade to Docker 1.1.0
 ANNOUNCE_TEMPLATE = """
 [Unit]
 Description={name} announce
@@ -209,7 +215,7 @@ def attach(self, name):
 
 [Service]
 EnvironmentFile=/etc/environment
-ExecStartPre=/bin/sh -c "until docker inspect -f '{{{{range $i, $e := .HostConfig.PortBindings }}}}{{{{$p := index $e 0}}}}{{{{$p.HostPort}}}}{{{{end}}}}' {name} >/dev/null 2>&1; do sleep 2; done; port=$(docker inspect -f '{{{{range $i, $e := .HostConfig.PortBindings }}}}{{{{$p := index $e 0}}}}{{{{$p.HostPort}}}}{{{{end}}}}' {name}); echo Waiting for $port/tcp...; until netstat -lnt | grep :$port >/dev/null; do sleep 1; done"
+ExecStartPre=/bin/sh -c "until docker inspect -f '{{{{range $i, $e := .HostConfig.PortBindings }}}}{{{{$p := index $e 0}}}}{{{{$p.HostPort}}}}{{{{end}}}}' {name} >/dev/null 2>&1; do sleep 2; done; port=$(docker inspect -f '{{{{range $i, $e := .HostConfig.PortBindings }}}}{{{{$p := index $e 0}}}}{{{{$p.HostPort}}}}{{{{end}}}}' {name}); if [[ -z $port ]]; then echo We have no port...; exit 1; fi; echo Waiting for $port/tcp...; until netstat -lnt | grep :$port >/dev/null; do sleep 1; done"
 ExecStart=/bin/sh -c "port=$(docker inspect -f '{{{{range $i, $e := .HostConfig.PortBindings }}}}{{{{$p := index $e 0}}}}{{{{$p.HostPort}}}}{{{{end}}}}' {name}); echo Connected to $COREOS_PRIVATE_IPV4:$port/tcp, publishing to etcd...; while netstat -lnt | grep :$port >/dev/null; do etcdctl set /deis/services/{app}/{name} $COREOS_PRIVATE_IPV4:$port --ttl 60 >/dev/null; sleep 45; done"
 ExecStop=/usr/bin/etcdctl rm --recursive /deis/services/{app}/{name}
 


Test patch: