Repository: deis/deis

Issue:
Support both Dockerfiles and Procfiles
Currently, deploying any codebase that contains both a Procfile and a Dockerfile will receive an error:

```
name 'slug_path' is not defined
```

This is because the variable `slug_path` is not defined for the relevant context in Python (see `builder/templates/builder` around line 60).

Ideally, a Dockerfile-based app with a Procfile should be treated exactly the same as a buildpack-based app with a Procfile. That is, the commands in the Procfile should be considered authoritative, and supercede the CMD in the Dockerfile.

So just like with buildpacks, the same build output (in this case a container image, not a slug) can still be run with the command from the Procfile, right? The only issue I can think of might be that the Procfile format (with which I'm pretty unfamiliar) may allow sequences of commands (or worse, I/O redirection) that won't work as a custom run argument to the docker executable.

But I've had reasonable success applying arbitrary shell commands to random containers, even if i have to wrap it in Bash like:

```
CMD ["bash", "-c", "bundle exec rake release && exec bundle exec rackup -p $PORT"]
```

Is this feature possible?



Discussion:
If I understand it correctly, [Docker deploys](http://docs.deis.io/en/v0.6.0/developer/dockerfile/) use a kind of special buildpack which is driven exclusively by the Dockerfile. The Dockerfile is all one needs to build a Docker container. Thats why I don't see any need for a Procfile here.

In my opinion Deis should ignore the Procfile if there is a Dockerfile and issue a warning.

At the very least we need to issue a warning or friendly error so Dockerfile + Procfile doesn't fail as it does today.  To be clear, let me describe the current logic...

If a Procfile exists and there is no Dockerfile, we run the relevant buildpack.  See:
https://github.com/deis/builder/blob/master/templates/builder#L58:L82

We then use a small Dockerfile-shim to produce a Docker image from the resulting tarball/slug:
https://github.com/deis/builder/blob/master/templates/builder#L28:L31

Whether built from a Dockerfile or a Procfile/buildpack, in all cases the result is a Docker image.

The fact that Dockerfile based deploys _currently_ ignore the Procfile is not a technical necessity, it's just a consequence of the current Python code in `builder/templates/builder`. Parsing the Procfile for Dockerfile-based deployments should be possible, and is precisely what I am requesting as a feature.

The point of the Procfile is so that different types of commands, like worker processes as well as HTTP listeners, can be run from the same build artifact. This need is no less present for Dockerfile-based apps! Whether Deis uses a buildpack to achieve this is an implementation issue, on which I have no opinion.

This feature request is to support multiple processs types for Dockerfile-based Deis apps, just as for non-Dockerfile based apps. Without this feature in place, users with Dockerfile-based apps will have to create a separate Deis app for every process type, and push a separately-maintained branch of their codebase just to achieve what non-Dockerfile-based apps get for free. This will substantially discourage adoption for this type of app, in my opinion. (I am evaluating this product on behalf of a fairly large company running several hundred concurrent AWS instances).

> The point of the Procfile is so that different types of commands [...]

@benton You are right. This feature is missing from Docker deploys. Now I understand the use case. You want to use the same Dockerfile with different commands running according the process types.

How do you like the idea to be able to use multiple Docker files - one for each process type? If you could, you would be even more flexible. You could build a Docker container for `web` using runtime A and another Docker container for `worker` using runtime B. The names of the Dockerfiles could be `Dockerfile_web` and `Dockerfile_worker`. It would also make sense to reserve a process type `base` for a base Dockerfile which can be used in the others `FROM` line.

@alexanderkiel Thanks for your comments.

If I had to choose between the two workflows in question, I guess I'd rather have the Procfile used in both cases, but I recognize that as an institutional preference. Many of the developers on our app teams have used Heroku and and currently use Procfiles with foreman on their workstations. So this means that many of our repos (though not a majority) already have Procfiles.

Also, it makes the process definitions exactly the same for both Dockerfile and non-Dockerfile Deis apps, which is desirable just from a uniformity perspective.

Separate Dockerfiles would also be a reasonable workflow -- but not as DRY, because the contents of all the Dockerfiles is going to be the same except for the CMD line.

All that said, I'd be happy to have the feature implemented either way. I'm not much of a Python programmer, but will happily test and debug.

I submitted a PR to Docker a while back to address this issue through the use of a new TAG field in Dockerfiles.  See dotcloud/docker#3251.  Over time, I've soured on that approach and instead would prefer to have Procfiles respected on their own.

I think there may be an opportunity to add this support outside the builder and instead in the runtime component.  We should be able to write a container executor that doesn't need to know in advance a) the defined Procfile process types or b) their exact command syntax.

For example, if you issue a `deis scale web=8 worker=2` we can probably assume a) you have a Procfile b) you defined web and worker commands.  If you scale `unknown=3` and that is not defined in the Procfile (or if there is no Procfile), the container execution can fail with an error.

This leaves open the question of how to scale Docker containers that do not have a Procfile (or process type equivalent).  Today we use `web` as the default process type, but I think that's a misleading default.  Maybe `deis scale 4` (with a process type of None) is more appropriate?

Happy to hear more thoughts on this subject, as I've been wrestling with it for many months now!

@gabrtv My thoughts exactly. Multiple CMDs or alternate CMD directives in a Dockerfile is kind against the "one container one process" philosophy (which of course is just that -- the Phusion guys have a whole suite of Docker images based on runit!). Procfiles seem to be emerging as a de-facto standard place for this "type of work within a 12-factor codebase" info.

I also agree that your builder should (hopefully) not need to be changed. The runtime should be able to override the CMD, exactly as happens when you do `deis run ...`. So perhaps it would not be too onerous a feature to implement.  :)


Resolving patch:
diff --git a/controller/api/models.py b/controller/api/models.py
--- a/controller/api/models.py
+++ b/controller/api/models.py
@@ -257,7 +257,7 @@ def _get_scheduler(self):
     def _get_command(self):
         c_type = self.type
         if c_type:
-            return 'start {c_type}'
+            return "cat Procfile | grep ^{c_type} | cut -f 1 -d ' ' --complement | sh -"
         else:
             return ''
 


Test patch: