Cleanup: Refactor Dockerfile parser

This completely rewrites the Dockerfile parser/evaluator. Most logic is not changed and all tests pass related to the dockerfile handling (including building docker itself). See below for caveats. I will squash this after the review is done unless requested otherwise. I realize the commits are kind of a mess right now.

**This is a huge change and I need testers and broken Dockerfiles.**

It implements two new packages: `builder/parser` and `builder/evaluator`. The parser parses the Dockerfile, and the evaluator evaluates it and executes the instructions. Both use a dispatcher system to hook statement parsing and evaluation of parsed statements. The result is a clean, extensible interface to both, one can adjust the syntax of the Dockerfile without affecting the processing, or at least if necessary will require minimal adjustment.

`builder/parser/dumper` can be installed with `go get` and runs against a dockerfile and outputs the statements in s-expression format.

**CHANGES**:

* env handling is across all statements now. `WORKDIR`, `VOLUME`, even `ENTRYPOINT` can interpolate environment variables now. There have been a lot of requests for this in the issue tracker, but the old parser was too brittle to support it. I took the liberty of implementing this change.
* ONBUILD structure changes. This ends up in no different functionality, but because of the way it is dispatched, "step" notifications for ONBUILD instructions have their own effective "stack" and start at step 0 in the output.

**SUGGESTIONS FOR ADDITIONAL REFACTORING**:

I made no attempt to add new functionality because I did not want to cloud the PR. However, I think these things should be done:

* `builder/evaluator/internals.go` needs to be broken out into another library, and the functionality needs to be partitioned cleaner.
* features: `docker build --debug` should be a thing to output the parser result, and `docker build --shell` could allow us to provide a REPL to building docker images without `docker commit`.
* syntax transformations: it is more possible now than ever to manipulate how dockerfiles are parsed and evaluated as separate entities, we should at least discuss the option of higher level DSLs which can inject themselves into this process. Related, statement proposals like `INCLUDE` and `IN` can be functions of hte parser and the evaluator can mostly be left alone.

Last but not least, given the impact of this change I would like to get it in soon, or block other builder edits until it gets in. I can port simple requests into this PR instead.
Haha. Well, I figured it was better to use a type instead of the same construct everywhere. I can make it private (it couldn’t be before the builder move), but I’d like to keep it.

-Erik

> On Aug 15, 2014, at 10:26 AM, Alexandr Morozov <notifications@github.com> wrote:
> 
> In builder/evaluator.go:
> 
> > +	"io"
> > +	"io/ioutil"
> > +	"os"
> > +	"path"
> > +	"strings"
> > +
> > +	"github.com/docker/docker/builder/parser"
> > +	"github.com/docker/docker/daemon"
> > +	"github.com/docker/docker/engine"
> > +	"github.com/docker/docker/pkg/tarsum"
> > +	"github.com/docker/docker/registry"
> > +	"github.com/docker/docker/runconfig"
> > +	"github.com/docker/docker/utils"
> > +)
> > +
> > +type UniqueMap map[string]struct{}
> Pretty strange name for set, also public. I was confused about this, can we just write map[string]struct{}? Little verbose but code reader will not need to look what the hell is UniqueMap.
> 
> —
> Reply to this email directly or view it on GitHub <https://github.com/docker/docker/pull/7461/files#r16304045>.
>