Repository: roboll/helmfile
Pull request #1288 (2020-05-30 06:02:46)

Add integration test for Kustomize inetgration
### Context

I was working on a helmfile.yaml that applies json patches to a chart that uses `{{ .Release.Namespace }}` in its templates. ([kube-state-metrics to be precise](https://github.com/helm/charts/blob/ed7c211dc21f8f4435c802782a76ffdf4309e400/stable/kube-state-metrics/templates/_helpers.tpl#L45).)

### What happened
When I applied it, its value was set to the kube context that was active at the time. 

### Expected
As the helmfile.yaml had explicitly specified the namespace for the chart, I expected `{{ .Release.Namespace }}` to be resolved to that namespace, not the context's.

### Notes
I'm new to Go and have a feeling that storing a helper 'script' under `test/` and building it as an executable may not be considered best practice. So: please know that this is coming from a place of ignorance.

Also: will not force-push anymore after I mark this as ready for review (unless there's request to squash commits).

I'd be happy to split out the parts that add assertions on the result of `helmfile template` to a separate PR, if that'd be easier to review. I've kept the commits that show my journey through helm-diff, kustomize, and the official yaml package so that it's easier to follow why I ended up with the current implementation.


Patch:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 780ad78..5050643 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -16,6 +16,7 @@ jobs:
           - go-mod-cache-v1-
       - run: go env
       - run: make build
+      - run: make build-test-tools
       - save_cache:
           key: go-mod-cache-v1-{{ checksum "./go.sum" }}
           paths:
@@ -87,7 +88,10 @@ jobs:
       - run: mkdir ~/build
       - attach_workspace:
           at: ~/build
-      - run: cp ~/build/helmfile ~/project/helmfile
+      - run:
+          command: |
+            cp ~/build/helmfile ~/project/helmfile
+            cp ~/build/diff-yamls ~/project/diff-yamls
       - run:
           name: Install helm
           environment:
@@ -98,6 +102,16 @@ jobs:
             tar zxf ${HELM_FILENAME} linux-amd64/helm
             chmod +x linux-amd64/helm
             sudo mv linux-amd64/helm /usr/local/bin/
+      - run:
+          name: Install kustomize
+          environment:
+            KUSTOMIZE_VERSION: v3.6.1
+          command: |
+            KUSTOMIZE_FILENAME="kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz"
+            curl -Lo ${KUSTOMIZE_FILENAME} "https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F${KUSTOMIZE_VERSION}/${KUSTOMIZE_FILENAME}"
+            tar zxf ${KUSTOMIZE_FILENAME} kustomize
+            chmod +x kustomize
+            sudo mv kustomize /usr/local/bin/
       - run:
           name: Deploy minikube
           environment:
@@ -132,7 +146,10 @@ jobs:
     - run: mkdir ~/build
     - attach_workspace:
         at: ~/build
-    - run: cp ~/build/helmfile ~/project/helmfile
+    - run:
+        command: |
+          cp ~/build/helmfile ~/project/helmfile
+          cp ~/build/diff-yamls ~/project/diff-yamls
     - run:
         name: Install helm
         environment:
@@ -143,6 +160,16 @@ jobs:
           tar zxf ${HELM_FILENAME} linux-amd64/helm
           chmod +x linux-amd64/helm
           sudo mv linux-amd64/helm /usr/local/bin/
+    - run:
+        name: Install kustomize
+        environment:
+          KUSTOMIZE_VERSION: v3.6.1
+        command: |
+          KUSTOMIZE_FILENAME="kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz"
+          curl -Lo ${KUSTOMIZE_FILENAME} "https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2F${KUSTOMIZE_VERSION}/${KUSTOMIZE_FILENAME}"
+          tar zxf ${KUSTOMIZE_FILENAME} kustomize
+          chmod +x kustomize
+          sudo mv kustomize /usr/local/bin/
     - run:
         name: Deploy minikube
         environment:
diff --git a/.gitignore b/.gitignore
index e3f7895..583077f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,4 +2,5 @@ dist/
 .idea/
 helmfile
 helmfile.lock
+test/integration/tmp
 vendor/
diff --git a/Makefile b/Makefile
index 0492ee4..0d2c3fc 100644
--- a/Makefile
+++ b/Makefile
@@ -17,6 +17,10 @@ check:
 	go vet ${PKGS}
 .PHONY: check
 
+build-test-tools:
+	go build test/diff-yamls.go
+.PHONY: build-test-tools
+
 test:
 	go test -v ${PKGS} -cover -race -p=1
 .PHONY: test
diff --git a/README.md b/README.md
index ba56b3f..89d027c 100644
--- a/README.md
+++ b/README.md
@@ -532,6 +532,7 @@ In addition to built-in ones, the following custom template functions are availa
 - `fromYaml` reads a golang string and generates a map
 - `setValueAtPath PATH NEW_VALUE` traverses a golang map, replaces the value at the PATH with NEW_VALUE
 - `toYaml` marshals a map into a string
+- `get` returns the value of the specified key if present in the `.Values` object, otherwise will return the default value defined in the function
 
 ### Values Files Templates
 
@@ -671,7 +672,7 @@ releaseName: prod
 `values.yaml.gotmpl`
 
 ```yaml
-domain: {{ .Values | getOrNil "my.domain" | default "dev.example.com" }}
+domain: {{ .Values | get "my.domain" "dev.example.com" }}
 ```
 
 `helmfile sync` installs `myapp` with the value `domain=dev.example.com`,
@@ -982,6 +983,19 @@ Now, replace `echo` with any command you like, and rewrite `args` that actually
 For templating, imagine that you created a hook that generates a helm chart on-the-fly by running an external tool like ksonnet, kustomize, or your own template engine.
 It will allow you to write your helm releases with any language you like, while still leveraging goodies provided by helm.
 
+### Global Hooks
+In contrast to the per release hooks mentioned above these are run only once at the very beginning and end of the execution of a helmfile command and only the `prepare` and `cleanup` hooks are available respectively.
+
+They use the same syntax as per release hooks, but at the top level of your helmfile:
+``` yaml
+hooks:
+- events: ["prepare", "cleanup"]
+  showlogs: true
+  command: "echo"
+  args: ["{{`{{.Environment.Name}}`}}", "{{`{{.Release.Name}}`}}", "{{`{{.HelmfileCommand}}`}}\
+"]
+```
+
 ### Helmfile + Kustomize
 
 Do you prefer `kustomize` to write and organize your Kubernetes apps, but still want to leverage helm's useful features
diff --git a/docs/writing-helmfile.md b/docs/writing-helmfile.md
index 792785e..4846bd9 100644
--- a/docs/writing-helmfile.md
+++ b/docs/writing-helmfile.md
@@ -14,10 +14,10 @@ That is, the following example let `helmfile` fail when you have no `eventApi.re
 {{ .Values.eventApi.replicas | default 1 }}
 ```
 
-In case it isn't a mistake and you do want to allow missing keys, use the `getOrNil` template function:
+In case it isn't a mistake and you do want to allow missing keys, use the `get` template function:
 
 ```
-{{ .Values | getOrNil "eventApi.replicas" }}
+{{ .Values | get "eventApi.replicas" nil }}
 ```
 
 This result in printing `<no value` in your template, that may or may not result in a failure.
@@ -25,7 +25,7 @@ This result in printing `<no value` in your template, that may or may not result
 If you want a kind of default values that is used when a missing key was referenced, use `default` like:
 
 ```
-{{ .Values | getOrNil "eventApi.replicas" | default 1 }}
+{{ .Values | get "eventApi.replicas" 1 }}
 ```
 
 Now, you get `1` when there is no `eventApi.replicas` defined in environment values.
diff --git a/go.mod b/go.mod
index b0d8ae3..ad39d9e 100644
--- a/go.mod
+++ b/go.mod
@@ -21,6 +21,7 @@ require (
 	github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
 	github.com/pierrec/lz4 v2.3.0+incompatible // indirect
 	github.com/r3labs/diff v0.0.0-20190801153147-a71de73c46ad
+	github.com/spf13/cobra v0.0.3
 	github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939
 	github.com/urfave/cli v1.20.0
 	github.com/variantdev/chartify v0.3.7
@@ -35,4 +36,5 @@ require (
 	gopkg.in/yaml.v2 v2.2.4
 	gotest.tools v2.2.0+incompatible
 	gotest.tools/v3 v3.0.3-0.20200410202438-4e4a41b7851a
+	k8s.io/apimachinery v0.0.0-20190409092423-760d1845f48b
 )
diff --git a/go.sum b/go.sum
index 1768b18..ae10366 100644
--- a/go.sum
+++ b/go.sum
@@ -650,8 +650,10 @@ github.com/ory/dockertest v3.3.4+incompatible/go.mod h1:1vX4m9wsvi00u5bseYwXaSnh
 github.com/otiai10/copy v1.1.1 h1:PH7IFlRQ6Fv9vYmuXbDRLdgTHoP1w483kPNUP2bskpo=
 github.com/otiai10/copy v1.1.1/go.mod h1:rrF5dJ5F0t/EWSYODDu4j9/vEeYHMkc8jt0zJChqQWw=
 github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95/go.mod h1:9qAhocn7zKJG+0mI8eUu6xqkFDYS2kb2saOteoSB3cE=
+github.com/otiai10/curr v1.0.0 h1:TJIWdbX0B+kpNagQrjgq8bCMrbhiuX73M2XwgtDMoOI=
 github.com/otiai10/curr v1.0.0/go.mod h1:LskTG5wDwr8Rs+nNQ+1LlxRjAtTZZjtJW4rMXl6j4vs=
 github.com/otiai10/mint v1.3.0/go.mod h1:F5AjcsTsWUqX+Na9fpHb52P8pcRX2CI6A3ctIT91xUo=
+github.com/otiai10/mint v1.3.1 h1:BCmzIS3n71sGfHB5NMNDB3lHYPz8fWSkCAErHed//qc=
 github.com/otiai10/mint v1.3.1/go.mod h1:/yxELlJQ0ufhjUwhshSj+wFjZ78CnZ48/1wtmBH1OTc=
 github.com/oxtoacart/bpool v0.0.0-20150712133111-4e1c5567d7c2 h1:CXwSGu/LYmbjEab5aMCs5usQRVBGThelUKBNnoSOuso=
 github.com/oxtoacart/bpool v0.0.0-20150712133111-4e1c5567d7c2/go.mod h1:L3UMQOThbttwfYRNFOWLLVXMhk5Lkio4GGOtw5UrxS0=
@@ -753,6 +755,7 @@ github.com/smartystreets/goconvey v0.0.0-20190710185942-9d28bd7c0945/go.mod h1:s
 github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
 github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d/go.mod h1:UdhH50NIW0fCiwBSr0co2m7BnFLdv4fQTgdqdJTHFeE=
 github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e/go.mod h1:HuIsMU8RRBOtsCgI77wP899iHVBQpCmg4ErYMZB+2IA=
+github.com/spf13/cobra v0.0.3 h1:ZlrZ4XsMRm04Fr5pSFxBgfND2EBVa1nLpiy1stUsX/8=
 github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
 github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
 github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
@@ -1050,6 +1053,7 @@ google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/
 google.golang.org/api v0.20.0 h1:jz2KixHX7EcCPiQrySzPdnYT7DbINAypCqKZ1Z7GM40=
 google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
 google.golang.org/api v0.21.0 h1:zS+Q/CJJnVlXpXQVIz+lH0ZT2lBuT2ac7XD8Y/3w6hY=
+google.golang.org/api v0.26.0 h1:VJZ8h6E8ip82FRpQl848c5vAadxlTXrUh8RzQzSRm08=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.3.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
diff --git a/pkg/app/app.go b/pkg/app/app.go
index 294379e..817fb74 100644
--- a/pkg/app/app.go
+++ b/pkg/app/app.go
@@ -38,6 +38,7 @@ type App struct {
 	FileOrDir string
 
 	readFile          func(string) ([]byte, error)
+	deleteFile        func(string) error
 	fileExists        func(string) (bool, error)
 	glob              func(string) ([]string, error)
 	abs               func(string) (string, error)
@@ -82,6 +83,7 @@ func New(conf ConfigProvider) *App {
 
 func Init(app *App) *App {
 	app.readFile = ioutil.ReadFile
+	app.deleteFile = os.Remove
 	app.glob = filepath.Glob
 	app.abs = filepath.Abs
 	app.getwd = os.Getwd
@@ -101,7 +103,7 @@ func Init(app *App) *App {
 
 func (a *App) Deps(c DepsConfigProvider) error {
 	return a.ForEachStateFiltered(func(run *Run) (errs []error) {
-		prepErrs := run.withReposAndPreparedCharts(false, c.SkipRepos(), func() {
+		prepErrs := run.withReposAndPreparedCharts(false, c.SkipRepos(), "deps", func() {
 			errs = run.Deps(c)
 		})
 
@@ -113,7 +115,7 @@ func (a *App) Deps(c DepsConfigProvider) error {
 
 func (a *App) Repos(c ReposConfigProvider) error {
 	return a.ForEachStateFiltered(func(run *Run) (errs []error) {
-		err := run.withPreparedCharts(false, func() {
+		err := run.withPreparedCharts(false, "repos", func() {
 			errs = run.Repos(c)
 		})
 
@@ -127,7 +129,7 @@ func (a *App) Repos(c ReposConfigProvider) error {
 
 func (a *App) DeprecatedSyncCharts(c DeprecatedChartsConfigProvider) error {
 	return a.ForEachStateFiltered(func(run *Run) (errs []error) {
-		err := run.withPreparedCharts(false, func() {
+		err := run.withPreparedCharts(false, "charts", func() {
 			errs = run.DeprecatedSyncCharts(c)
 		})
 
@@ -153,7 +155,7 @@ func (a *App) Diff(c DiffConfigProvider) error {
 
 		var errs []error
 
-		prepErrs := run.withReposAndPreparedCharts(false, c.SkipDeps(), func() {
+		prepErrs := run.withReposAndPreparedCharts(false, c.SkipDeps(), "diff", func() {
 			msg, matched, affected, errs = run.Diff(c)
 		})
 
@@ -193,7 +195,7 @@ func (a *App) Diff(c DiffConfigProvider) error {
 		// See https://github.com/roboll/helmfile/issues/749
 		code := 2
 		e := &Error{
-			msg:  "Identified at least on change",
+			msg:  "Identified at least one change",
 			code: &code,
 		}
 		return e
@@ -204,7 +206,7 @@ func (a *App) Diff(c DiffConfigProvider) error {
 
 func (a *App) Template(c TemplateConfigProvider) error {
 	return a.ForEachState(func(run *Run) (ok bool, errs []error) {
-		prepErrs := run.withReposAndPreparedCharts(true, c.SkipDeps(), func() {
+		prepErrs := run.withReposAndPreparedCharts(true, c.SkipDeps(), "template", func() {
 			ok, errs = a.template(run, c)
 		})
 
@@ -216,7 +218,7 @@ func (a *App) Template(c TemplateConfigProvider) error {
 
 func (a *App) Lint(c LintConfigProvider) error {
 	return a.ForEachStateFiltered(func(run *Run) (errs []error) {
-		prepErrs := run.withReposAndPreparedCharts(true, c.SkipDeps(), func() {
+		prepErrs := run.withReposAndPreparedCharts(true, c.SkipDeps(), "lint", func() {
 			errs = run.Lint(c)
 		})
 
@@ -228,7 +230,7 @@ func (a *App) Lint(c LintConfigProvider) error {
 
 func (a *App) Sync(c SyncConfigProvider) error {
 	return a.ForEachState(func(run *Run) (ok bool, errs []error) {
-		prepErrs := run.withReposAndPreparedCharts(false, c.SkipDeps(), func() {
+		prepErrs := run.withReposAndPreparedCharts(false, c.SkipDeps(), "sync", func() {
 			ok, errs = a.sync(run, c)
 		})
 
@@ -248,7 +250,7 @@ func (a *App) Apply(c ApplyConfigProvider) error {
 	opts = append(opts, SetRetainValuesFiles(c.RetainValuesFiles()))
 
 	err := a.ForEachState(func(run *Run) (ok bool, errs []error) {
-		prepErrs := run.withReposAndPreparedCharts(false, c.SkipDeps(), func() {
+		prepErrs := run.withReposAndPreparedCharts(false, c.SkipDeps(), "apply", func() {
 			matched, updated, es := a.apply(run, c)
 
 			mut.Lock()
@@ -278,7 +280,7 @@ func (a *App) Apply(c ApplyConfigProvider) error {
 
 func (a *App) Status(c StatusesConfigProvider) error {
 	return a.ForEachStateFiltered(func(run *Run) (errs []error) {
-		err := run.withPreparedCharts(false, func() {
+		err := run.withPreparedCharts(false, "status", func() {
 			errs = run.Status(c)
 		})
 
@@ -292,7 +294,7 @@ func (a *App) Status(c StatusesConfigProvider) error {
 
 func (a *App) Delete(c DeleteConfigProvider) error {
 	return a.ForEachState(func(run *Run) (ok bool, errs []error) {
-		err := run.withPreparedCharts(false, func() {
+		err := run.withPreparedCharts(false, "delete", func() {
 			ok, errs = a.delete(run, c.Purge(), c)
 		})
 
@@ -306,7 +308,7 @@ func (a *App) Delete(c DeleteConfigProvider) error {
 
 func (a *App) Destroy(c DestroyConfigProvider) error {
 	return a.ForEachState(func(run *Run) (ok bool, errs []error) {
-		err := run.withPreparedCharts(false, func() {
+		err := run.withPreparedCharts(false, "destroy", func() {
 			ok, errs = a.delete(run, true, c)
 		})
 
@@ -326,7 +328,7 @@ func (a *App) Test(c TestConfigProvider) error {
 				"or set helm.sh/hook-delete-policy\n")
 		}
 
-		err := run.withPreparedCharts(false, func() {
+		err := run.withPreparedCharts(false, "test", func() {
 			errs = run.Test(c)
 		})
 
@@ -340,7 +342,7 @@ func (a *App) Test(c TestConfigProvider) error {
 
 func (a *App) PrintState(c StateConfigProvider) error {
 	return a.VisitDesiredStatesWithReleasesFiltered(a.FileOrDir, func(st *state.HelmState) (errs []error) {
-		err := NewRun(st, nil, NewContext()).withPreparedCharts(false, func() {
+		err := NewRun(st, nil, NewContext()).withPreparedCharts(false, "build", func() {
 			state, err := st.ToYaml()
 			if err != nil {
 				errs = []error{err}
@@ -363,7 +365,7 @@ func (a *App) ListReleases(c ListConfigProvider) error {
 	var releases []*HelmRelease
 
 	err := a.VisitDesiredStatesWithReleasesFiltered(a.FileOrDir, func(st *state.HelmState) []error {
-		err := NewRun(st, nil, NewContext()).withPreparedCharts(false, func() {
+		err := NewRun(st, nil, NewContext()).withPreparedCharts(false, "list", func() {
 
 			//var releases m
 			for _, r := range st.Releases {
@@ -482,11 +484,13 @@ func (a *App) loadDesiredStateFromYaml(file string, opts ...LoadOpts) (*state.He
 
 	ld := &desiredStateLoader{
 		readFile:   a.readFile,
+		deleteFile: a.deleteFile,
 		fileExists: a.fileExists,
 		env:        a.Env,
 		namespace:  a.Namespace,
 		logger:     a.Logger,
 		abs:        a.abs,
+		remote:     a.remote,
 
 		overrideKubeContext: a.OverrideKubeContext,
 		overrideHelmBinary:  a.OverrideHelmBinary,
@@ -791,18 +795,7 @@ func (a *App) visitStatesWithSelectorsAndRemoteSupport(fileOrDir string, converg
 		return err
 	}
 
-	getter := &remote.GoGetter{Logger: a.Logger}
-
-	remote := &remote.Remote{
-		Logger:     a.Logger,
-		Home:       dir,
-		Getter:     getter,
-		ReadFile:   a.readFile,
-		DirExists:  a.directoryExistsAt,
-		FileExists: a.fileExistsAt,
-	}
-
-	a.remote = remote
+	a.remote = remote.NewRemote(a.Logger, dir, a.readFile, a.directoryExistsAt, a.fileExistsAt)
 
 	return a.visitStates(fileOrDir, opts, converge)
 }
diff --git a/pkg/app/desired_state_file_loader.go b/pkg/app/desired_state_file_loader.go
index 3fee0cb..3d7a7df 100644
--- a/pkg/app/desired_state_file_loader.go
+++ b/pkg/app/desired_state_file_loader.go
@@ -5,6 +5,7 @@ import (
 	"errors"
 	"fmt"
 	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/remote"
 	"path/filepath"
 	"sort"
 
@@ -27,11 +28,13 @@ type desiredStateLoader struct {
 	namespace string
 
 	readFile   func(string) ([]byte, error)
+	deleteFile func(string) error
 	fileExists func(string) (bool, error)
 	abs        func(string) (string, error)
 	glob       func(string) ([]string, error)
 	getHelm    func(*state.HelmState) helmexec.Interface
 
+	remote      *remote.Remote
 	logger      *zap.SugaredLogger
 	valsRuntime vals.Evaluator
 }
@@ -46,7 +49,7 @@ func (ld *desiredStateLoader) Load(f string, opts LoadOpts) (*state.HelmState, e
 			return nil, fmt.Errorf("bug: opts.CalleePath was nil: f=%s, opts=%v", f, opts)
 		}
 		storage := state.NewStorage(opts.CalleePath, ld.logger, ld.glob)
-		envld := state.NewEnvironmentValuesLoader(storage, ld.readFile, ld.logger)
+		envld := state.NewEnvironmentValuesLoader(storage, ld.readFile, ld.logger, ld.remote)
 		handler := state.MissingFileHandlerError
 		vals, err := envld.LoadEnvironmentValues(&handler, args)
 		if err != nil {
@@ -147,7 +150,8 @@ func (ld *desiredStateLoader) loadFileWithOverrides(inheritedEnv, overrodeEnv *e
 }
 
 func (a *desiredStateLoader) underlying() *state.StateCreator {
-	c := state.NewCreator(a.logger, a.readFile, a.fileExists, a.abs, a.glob, a.valsRuntime, a.getHelm, a.overrideHelmBinary)
+	c := state.NewCreator(a.logger, a.readFile, a.fileExists, a.abs, a.glob, a.valsRuntime, a.getHelm, a.overrideHelmBinary, a.remote)
+	c.DeleteFile = a.deleteFile
 	c.LoadFile = a.loadFile
 	return c
 }
diff --git a/pkg/app/run.go b/pkg/app/run.go
index 0895ec0..2e3f2eb 100644
--- a/pkg/app/run.go
+++ b/pkg/app/run.go
@@ -32,7 +32,7 @@ func (r *Run) askForConfirmation(msg string) bool {
 	return AskForConfirmation(msg)
 }
 
-func (r *Run) withReposAndPreparedCharts(forceDownload bool, skipRepos bool, f func()) []error {
+func (r *Run) withReposAndPreparedCharts(forceDownload bool, skipRepos bool, helmfileCommand string, f func()) []error {
 	if !skipRepos {
 		ctx := r.ctx
 		if errs := ctx.SyncReposOnce(r.state, r.helm); errs != nil && len(errs) > 0 {
@@ -40,14 +40,14 @@ func (r *Run) withReposAndPreparedCharts(forceDownload bool, skipRepos bool, f f
 		}
 	}
 
-	if err := r.withPreparedCharts(forceDownload, f); err != nil {
+	if err := r.withPreparedCharts(forceDownload, helmfileCommand, f); err != nil {
 		return []error{err}
 	}
 
 	return nil
 }
 
-func (r *Run) withPreparedCharts(forceDownload bool, f func()) error {
+func (r *Run) withPreparedCharts(forceDownload bool, helmfileCommand string, f func()) error {
 	if r.ReleaseToChart != nil {
 		panic("Run.PrepareCharts can be called only once")
 	}
@@ -59,7 +59,11 @@ func (r *Run) withPreparedCharts(forceDownload bool, f func()) error {
 	}
 	defer os.RemoveAll(dir)
 
-	releaseToChart, errs := state.PrepareCharts(r.helm, r.state, dir, 2, "template", forceDownload)
+	if _, err = r.state.TriggerGlobalPrepareEvent(helmfileCommand); err != nil {
+		return err
+	}
+
+	releaseToChart, errs := r.state.PrepareCharts(r.helm, dir, 2, helmfileCommand, forceDownload)
 
 	if len(errs) > 0 {
 		return fmt.Errorf("%v", errs)
@@ -75,7 +79,9 @@ func (r *Run) withPreparedCharts(forceDownload bool, f func()) error {
 
 	f()
 
-	return nil
+	_, err = r.state.TriggerGlobalCleanupEvent(helmfileCommand)
+
+	return err
 }
 
 func (r *Run) Deps(c DepsConfigProvider) []error {
diff --git a/pkg/helmexec/exit_error.go b/pkg/helmexec/exit_error.go
index 9419cf4..31eaf7c 100644
--- a/pkg/helmexec/exit_error.go
+++ b/pkg/helmexec/exit_error.go
@@ -5,19 +5,55 @@ import (
 	"strings"
 )
 
-func newExitError(helmCmdPath string, exitStatus int, errorMessage string) ExitError {
+func newExitError(path string, args []string, exitStatus int, err error, stderr, combined string) ExitError {
+	var out string
+
+	out += fmt.Sprintf("PATH:\n%s", Indent(path, "  "))
+
+	out += "\n\nARGS:"
+	for i, a := range args {
+		out += fmt.Sprintf("\n%s", Indent(fmt.Sprintf("%d: %s (%d bytes)", i, a, len(a)), "  "))
+	}
+
+	out += fmt.Sprintf("\n\nERROR:\n%s", Indent(err.Error(), "  "))
+
+	out += fmt.Sprintf("\n\nEXIT STATUS\n%s", Indent(fmt.Sprintf("%d", exitStatus), "  "))
+
+	if len(stderr) > 0 {
+		out += fmt.Sprintf("\n\nSTDERR:\n%s", Indent(stderr, "  "))
+	}
+
+	if len(combined) > 0 {
+		out += fmt.Sprintf("\n\nCOMBINED OUTPUT:\n%s", Indent(combined, "  "))
+	}
+
 	return ExitError{
-		Message: fmt.Sprintf("the following cmd exited with status %d:\n%s\n\n%s", exitStatus, indent(strings.TrimSpace(helmCmdPath)), indent(strings.TrimSpace(errorMessage))),
+		Message: fmt.Sprintf("command %q exited with non-zero status:\n\n%s", path, out),
 		Code:    exitStatus,
 	}
 }
 
-func indent(text string) string {
+// indents a block of text with an indent string
+func Indent(text, indent string) string {
+	var b strings.Builder
+
+	b.Grow(len(text) * 2)
+
 	lines := strings.Split(text, "\n")
-	for i := range lines {
-		lines[i] = "  " + lines[i]
+
+	last := len(lines) - 1
+
+	for i, j := range lines {
+		if i > 0 && i < last && j != "" {
+			b.WriteString("\n")
+		}
+
+		if j != "" {
+			b.WriteString(indent + j)
+		}
 	}
-	return strings.Join(lines, "\n")
+
+	return b.String()
 }
 
 // ExitError is created whenever your shell command exits with a non-zero exit status
diff --git a/pkg/helmexec/runner.go b/pkg/helmexec/runner.go
index 179c019..8762c11 100644
--- a/pkg/helmexec/runner.go
+++ b/pkg/helmexec/runner.go
@@ -5,6 +5,7 @@ import (
 	"errors"
 	"fmt"
 	"go.uber.org/zap"
+	"io"
 	"os"
 	"os/exec"
 	"strings"
@@ -33,10 +34,10 @@ func (shell ShellRunner) Execute(cmd string, args []string, env map[string]strin
 	preparedCmd := exec.Command(cmd, args...)
 	preparedCmd.Dir = shell.Dir
 	preparedCmd.Env = mergeEnv(os.Environ(), env)
-	return combinedOutput(preparedCmd, shell.Logger)
+	return Output(preparedCmd)
 }
 
-func combinedOutput(c *exec.Cmd, logger *zap.SugaredLogger) ([]byte, error) {
+func Output(c *exec.Cmd) ([]byte, error) {
 	if c.Stdout != nil {
 		return nil, errors.New("exec: Stdout already set")
 	}
@@ -45,13 +46,11 @@ func combinedOutput(c *exec.Cmd, logger *zap.SugaredLogger) ([]byte, error) {
 	}
 	var stdout bytes.Buffer
 	var stderr bytes.Buffer
-	c.Stdout = &stdout
-	c.Stderr = &stderr
+	var combined bytes.Buffer
+	c.Stdout = io.MultiWriter(&stdout, &combined)
+	c.Stderr = io.MultiWriter(&stderr, &combined)
 	err := c.Run()
 
-	o := stdout.Bytes()
-	e := stderr.Bytes()
-
 	if err != nil {
 		// TrimSpace is necessary, because otherwise helmfile prints the redundant new-lines after each error like:
 		//
@@ -64,14 +63,13 @@ func combinedOutput(c *exec.Cmd, logger *zap.SugaredLogger) ([]byte, error) {
 			// so that helmfile could return its own exit code accordingly
 			waitStatus := ee.Sys().(syscall.WaitStatus)
 			exitStatus := waitStatus.ExitStatus()
-			cmd := fmt.Sprintf("%s %s", c.Path, strings.Join(c.Args, " "))
-			err = newExitError(cmd, exitStatus, string(e))
+			err = newExitError(c.Path, c.Args, exitStatus, ee, stderr.String(), combined.String())
 		default:
 			panic(fmt.Sprintf("unexpected error: %v", err))
 		}
 	}
 
-	return o, err
+	return stdout.Bytes(), err
 }
 
 func mergeEnv(orig []string, new map[string]string) []string {
diff --git a/pkg/remote/remote.go b/pkg/remote/remote.go
index ba4f118..e3e9179 100644
--- a/pkg/remote/remote.go
+++ b/pkg/remote/remote.go
@@ -260,3 +260,15 @@ func (g *GoGetter) Get(wd, src, dst string) error {
 
 	return nil
 }
+
+func NewRemote(logger *zap.SugaredLogger, homeDir string, readFile func(string) ([]byte, error), dirExists func(string) bool, fileExists func(string) bool) *Remote {
+	remote := &Remote{
+		Logger:     logger,
+		Home:       homeDir,
+		Getter:     &GoGetter{Logger: logger},
+		ReadFile:   readFile,
+		DirExists:  dirExists,
+		FileExists: fileExists,
+	}
+	return remote
+}
diff --git a/pkg/state/create.go b/pkg/state/create.go
index 67efa63..3d50488 100644
--- a/pkg/state/create.go
+++ b/pkg/state/create.go
@@ -5,6 +5,7 @@ import (
 	"errors"
 	"fmt"
 	"github.com/roboll/helmfile/pkg/helmexec"
+	"github.com/roboll/helmfile/pkg/remote"
 	"io"
 	"os"
 
@@ -43,6 +44,7 @@ type StateCreator struct {
 	fileExists  func(string) (bool, error)
 	abs         func(string) (string, error)
 	glob        func(string) ([]string, error)
+	DeleteFile  func(string) error
 	valsRuntime vals.Evaluator
 
 	Strict bool
@@ -52,9 +54,11 @@ type StateCreator struct {
 	getHelm func(*HelmState) helmexec.Interface
 
 	overrideHelmBinary string
+
+	remote *remote.Remote
 }
 
-func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), fileExists func(string) (bool, error), abs func(string) (string, error), glob func(string) ([]string, error), valsRuntime vals.Evaluator, getHelm func(*HelmState) helmexec.Interface, overrideHelmBinary string) *StateCreator {
+func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error), fileExists func(string) (bool, error), abs func(string) (string, error), glob func(string) ([]string, error), valsRuntime vals.Evaluator, getHelm func(*HelmState) helmexec.Interface, overrideHelmBinary string, remote *remote.Remote) *StateCreator {
 	return &StateCreator{
 		logger:      logger,
 		readFile:    readFile,
@@ -66,6 +70,8 @@ func NewCreator(logger *zap.SugaredLogger, readFile func(string) ([]byte, error)
 		getHelm:     getHelm,
 
 		overrideHelmBinary: overrideHelmBinary,
+
+		remote: remote,
 	}
 }
 
@@ -139,7 +145,7 @@ func (c *StateCreator) LoadEnvValues(target *HelmState, env string, ctxEnv *envi
 		return nil, &StateLoadError{fmt.Sprintf("failed to read %s", state.FilePath), err}
 	}
 
-	e.Defaults, err = state.loadValuesEntries(nil, state.DefaultValues)
+	e.Defaults, err = state.loadValuesEntries(nil, state.DefaultValues, c.remote)
 	if err != nil {
 		return nil, err
 	}
@@ -203,7 +209,7 @@ func (c *StateCreator) loadEnvValues(st *HelmState, name string, failOnMissingEn
 	envSpec, ok := st.Environments[name]
 	if ok {
 		var err error
-		envVals, err = st.loadValuesEntries(envSpec.MissingFileHandler, envSpec.Values)
+		envVals, err = st.loadValuesEntries(envSpec.MissingFileHandler, envSpec.Values, c.remote)
 		if err != nil {
 			return nil, err
 		}
@@ -276,6 +282,11 @@ func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles
 					results <- secretResult{nil, err, path}
 					continue
 				}
+				defer func() {
+					if err := c.DeleteFile(decFile); err != nil {
+						c.logger.Warnf("removing decrypted file %s: %w", decFile, err)
+					}
+				}()
 				bytes, err := readFile(decFile)
 				if err != nil {
 					results <- secretResult{nil, fmt.Errorf("failed to load environment secrets file \"%s\": %v", path, err), path}
@@ -321,11 +332,11 @@ func (c *StateCreator) scatterGatherEnvSecretFiles(st *HelmState, envSecretFiles
 	return nil
 }
 
-func (st *HelmState) loadValuesEntries(missingFileHandler *string, entries []interface{}) (map[string]interface{}, error) {
+func (st *HelmState) loadValuesEntries(missingFileHandler *string, entries []interface{}, remote *remote.Remote) (map[string]interface{}, error) {
 	envVals := map[string]interface{}{}
 
 	valuesEntries := append([]interface{}{}, entries...)
-	ld := NewEnvironmentValuesLoader(st.storage(), st.readFile, st.logger)
+	ld := NewEnvironmentValuesLoader(st.storage(), st.readFile, st.logger, remote)
 	var err error
 	envVals, err = ld.LoadEnvironmentValues(missingFileHandler, valuesEntries)
 	if err != nil {
diff --git a/pkg/state/envvals_loader.go b/pkg/state/envvals_loader.go
index c5ea8ef..53d605a 100644
--- a/pkg/state/envvals_loader.go
+++ b/pkg/state/envvals_loader.go
@@ -5,6 +5,7 @@ import (
 	"github.com/imdario/mergo"
 	"github.com/roboll/helmfile/pkg/environment"
 	"github.com/roboll/helmfile/pkg/maputil"
+	"github.com/roboll/helmfile/pkg/remote"
 	"github.com/roboll/helmfile/pkg/tmpl"
 	"go.uber.org/zap"
 	"gopkg.in/yaml.v2"
@@ -17,13 +18,16 @@ type EnvironmentValuesLoader struct {
 	readFile func(string) ([]byte, error)
 
 	logger *zap.SugaredLogger
+
+	remote *remote.Remote
 }
 
-func NewEnvironmentValuesLoader(storage *Storage, readFile func(string) ([]byte, error), logger *zap.SugaredLogger) *EnvironmentValuesLoader {
+func NewEnvironmentValuesLoader(storage *Storage, readFile func(string) ([]byte, error), logger *zap.SugaredLogger, remote *remote.Remote) *EnvironmentValuesLoader {
 	return &EnvironmentValuesLoader{
 		storage:  storage,
 		readFile: readFile,
 		logger:   logger,
+		remote:   remote,
 	}
 }
 
@@ -36,6 +40,11 @@ func (ld *EnvironmentValuesLoader) LoadEnvironmentValues(missingFileHandler *str
 		switch strOrMap := entry.(type) {
 		case string:
 			urlOrPath := strOrMap
+			localPath, err := ld.remote.Locate(urlOrPath)
+			if err == nil {
+				urlOrPath = localPath
+			}
+
 			files, skipped, err := ld.storage.resolveFile(missingFileHandler, "environment values", urlOrPath)
 			if err != nil {
 				return nil, err
diff --git a/pkg/state/helmx.go b/pkg/state/helmx.go
index fe4b36b..adaac66 100644
--- a/pkg/state/helmx.go
+++ b/pkg/state/helmx.go
@@ -22,7 +22,7 @@ func (st *HelmState) appendHelmXFlags(flags []string, release *ReleaseSpec) ([]s
 	return flags, nil
 }
 
-func (st *HelmState) PrepareChartify(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) (bool, *chartify.ChartifyOpts, error) {
+func (st *HelmState) PrepareChartify(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) (bool, *chartify.ChartifyOpts, func(), error) {
 	var opts chartify.ChartifyOpts
 
 	opts.WorkaroundOutputDirIssue = true
@@ -33,6 +33,8 @@ func (st *HelmState) PrepareChartify(helm helmexec.Interface, release *ReleaseSp
 
 	opts.ChartVersion = release.Version
 
+	opts.Namespace = release.Namespace
+
 	dir := filepath.Join(st.basePath, release.Chart)
 	if stat, _ := os.Stat(dir); stat != nil && stat.IsDir() {
 		if exists, err := st.fileExists(filepath.Join(dir, "Chart.yaml")); err == nil && !exists {
@@ -64,19 +66,25 @@ func (st *HelmState) PrepareChartify(helm helmexec.Interface, release *ReleaseSp
 		shouldRun = true
 	}
 
+	var filesNeedCleaning []string
+
+	clean := func() {
+		st.removeFiles(filesNeedCleaning)
+	}
+
 	jsonPatches := release.JSONPatches
 	if len(jsonPatches) > 0 {
 		generatedFiles, err := st.generateTemporaryValuesFiles(jsonPatches, release.MissingFileHandler)
 		if err != nil {
-			return false, nil, err
+			return false, nil, clean, err
 		}
 
+		filesNeedCleaning = append(filesNeedCleaning, generatedFiles...)
+
 		for _, f := range generatedFiles {
 			opts.JsonPatches = append(opts.JsonPatches, f)
 		}
 
-		release.generatedValues = append(release.generatedValues, generatedFiles...)
-
 		shouldRun = true
 	}
 
@@ -84,14 +92,14 @@ func (st *HelmState) PrepareChartify(helm helmexec.Interface, release *ReleaseSp
 	if len(strategicMergePatches) > 0 {
 		generatedFiles, err := st.generateTemporaryValuesFiles(strategicMergePatches, release.MissingFileHandler)
 		if err != nil {
-			return false, nil, err
+			return false, nil, clean, err
 		}
 
 		for _, f := range generatedFiles {
 			opts.StrategicMergePatches = append(opts.StrategicMergePatches, f)
 		}
 
-		release.generatedValues = append(release.generatedValues, generatedFiles...)
+		filesNeedCleaning = append(filesNeedCleaning, generatedFiles...)
 
 		shouldRun = true
 	}
@@ -99,11 +107,13 @@ func (st *HelmState) PrepareChartify(helm helmexec.Interface, release *ReleaseSp
 	if shouldRun {
 		generatedFiles, err := st.generateValuesFiles(helm, release, workerIndex)
 		if err != nil {
-			return false, nil, err
+			return false, nil, clean, err
 		}
 
+		filesNeedCleaning = append(filesNeedCleaning, generatedFiles...)
+
 		opts.ValuesFiles = generatedFiles
 	}
 
-	return shouldRun, &opts, nil
+	return shouldRun, &opts, clean, nil
 }
diff --git a/pkg/state/state.go b/pkg/state/state.go
index 21804a3..04643f1 100644
--- a/pkg/state/state.go
+++ b/pkg/state/state.go
@@ -59,6 +59,9 @@ type HelmState struct {
 	Selectors          []string          `yaml:"-"`
 	ApiVersions        []string          `yaml:"apiVersions,omitempty"`
 
+	// Hooks is a list of extension points paired with operations, that are executed in specific points of the lifecycle of releases defined in helmfile
+	Hooks []event.Hook `yaml:"hooks,omitempty"`
+
 	Templates map[string]TemplateSpec `yaml:"templates"`
 
 	Env environment.Environment `yaml:"-"`
@@ -216,8 +219,6 @@ type ReleaseSpec struct {
 	StrategicMergePatches []interface{} `yaml:"strategicMergePatches,omitempty"`
 	Adopt                 []string      `yaml:"adopt,omitempty"`
 
-	// generatedValues are values that need cleaned up on exit
-	generatedValues []string
 	//version of the chart that has really been installed cause desired version may be fuzzy (~2.0.0)
 	installedVersion string
 }
@@ -296,6 +297,7 @@ type syncPrepareResult struct {
 	release *ReleaseSpec
 	flags   []string
 	errors  []*ReleaseError
+	files   []string
 }
 
 // SyncReleases wrapper for executing helm upgrade on the releases
@@ -346,10 +348,10 @@ func (st *HelmState) prepareSyncReleases(helm helmexec.Interface, additionalValu
 				// TODO We need a long-term fix for this :)
 				// See https://github.com/roboll/helmfile/issues/737
 				mut.Lock()
-				flags, flagsErr := st.flagsForUpgrade(helm, release, workerIndex)
+				flags, files, flagsErr := st.flagsForUpgrade(helm, release, workerIndex)
 				mut.Unlock()
 				if flagsErr != nil {
-					results <- syncPrepareResult{errors: []*ReleaseError{newReleaseFailedError(release, flagsErr)}}
+					results <- syncPrepareResult{errors: []*ReleaseError{newReleaseFailedError(release, flagsErr)}, files: files}
 					continue
 				}
 
@@ -376,11 +378,11 @@ func (st *HelmState) prepareSyncReleases(helm helmexec.Interface, additionalValu
 				}
 
 				if len(errs) > 0 {
-					results <- syncPrepareResult{errors: errs}
+					results <- syncPrepareResult{errors: errs, files: files}
 					continue
 				}
 
-				results <- syncPrepareResult{release: release, flags: flags, errors: []*ReleaseError{}}
+				results <- syncPrepareResult{release: release, flags: flags, errors: []*ReleaseError{}, files: files}
 			}
 		},
 		func() {
@@ -568,6 +570,13 @@ func (st *HelmState) SyncReleases(affectedReleases *AffectedReleases, helm helme
 	}
 
 	preps, prepErrs := st.prepareSyncReleases(helm, additionalValues, workerLimit, opts)
+
+	defer func() {
+		for _, p := range preps {
+			st.removeFiles(p.files)
+		}
+	}()
+
 	if len(prepErrs) > 0 {
 		return prepErrs
 	}
@@ -713,7 +722,7 @@ func (st *HelmState) getDeployedVersion(context helmexec.HelmContext, helm helme
 // Otheriwse, if a chart is not a helm chart, it will call "chartify" to turn it into a chart.
 //
 // If exists, it will also patch resources by json patches, strategic-merge patches, and injectors.
-func PrepareCharts(helm helmexec.Interface, st *HelmState, dir string, concurrency int, helmfileCommand string, forceDownload bool) (map[string]string, []error) {
+func (st *HelmState) PrepareCharts(helm helmexec.Interface, dir string, concurrency int, helmfileCommand string, forceDownload bool) (map[string]string, []error) {
 	temp := make(map[string]string, len(st.Releases))
 	type downloadResults struct {
 		releaseName string
@@ -744,7 +753,8 @@ func PrepareCharts(helm helmexec.Interface, st *HelmState, dir string, concurren
 			for release := range jobQueue {
 				var chartPath string
 
-				shouldChartify, opts, err := st.PrepareChartify(helm, release, workerIndex)
+				shouldChartify, opts, clean, err := st.PrepareChartify(helm, release, workerIndex)
+				defer clean()
 				if err != nil {
 					results <- &downloadResults{err: err}
 					return
@@ -844,15 +854,20 @@ func (st *HelmState) TemplateReleases(helm helmexec.Interface, outputDir string,
 	}
 
 	for i := range st.Releases {
-		release := st.Releases[i]
+		release := &st.Releases[i]
 
 		if !release.Desired() {
 			continue
 		}
 
-		st.ApplyOverrides(&release)
+		st.ApplyOverrides(release)
+
+		flags, files, err := st.flagsForTemplate(helm, release, 0)
+
+		defer func() {
+			st.removeFiles(files)
+		}()
 
-		flags, err := st.flagsForTemplate(helm, &release, 0)
 		if err != nil {
 			errs = append(errs, err)
 		}
@@ -883,7 +898,10 @@ func (st *HelmState) TemplateReleases(helm helmexec.Interface, outputDir string,
 
 			flags = append(flags, "--output-dir", releaseOutputDir)
 			st.logger.Debugf("Generating templates to : %s\n", releaseOutputDir)
-			os.Mkdir(releaseOutputDir, 0755)
+			err = os.MkdirAll(releaseOutputDir, 0755)
+			if err != nil {
+				errs = append(errs, err)
+			}
 		}
 
 		if validate {
@@ -896,7 +914,7 @@ func (st *HelmState) TemplateReleases(helm helmexec.Interface, outputDir string,
 			}
 		}
 
-		if _, err := st.triggerCleanupEvent(&release, "template"); err != nil {
+		if _, err := st.triggerCleanupEvent(release, "template"); err != nil {
 			st.logger.Warnf("warn: %v\n", err)
 		}
 	}
@@ -941,7 +959,10 @@ func (st *HelmState) LintReleases(helm helmexec.Interface, additionalValues []st
 			continue
 		}
 
-		flags, err := st.flagsForLint(helm, &release, 0)
+		flags, files, err := st.flagsForLint(helm, &release, 0)
+
+		defer st.removeFiles(files)
+
 		if err != nil {
 			errs = append(errs, err)
 		}
@@ -989,6 +1010,7 @@ type diffPrepareResult struct {
 	release *ReleaseSpec
 	flags   []string
 	errors  []*ReleaseError
+	files   []string
 }
 
 func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValues []string, concurrency int, detailedExitCode, includeTests, suppressSecrets bool, opt ...DiffOpt) ([]diffPrepareResult, []error) {
@@ -1035,7 +1057,7 @@ func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValu
 				// TODO We need a long-term fix for this :)
 				// See https://github.com/roboll/helmfile/issues/737
 				mut.Lock()
-				flags, err := st.flagsForDiff(helm, release, workerIndex)
+				flags, files, err := st.flagsForDiff(helm, release, workerIndex)
 				mut.Unlock()
 				if err != nil {
 					errs = append(errs, err)
@@ -1084,9 +1106,9 @@ func (st *HelmState) prepareDiffReleases(helm helmexec.Interface, additionalValu
 					for i, e := range errs {
 						rsErrs[i] = newReleaseFailedError(release, e)
 					}
-					results <- diffPrepareResult{errors: rsErrs}
+					results <- diffPrepareResult{errors: rsErrs, files: files}
 				} else {
-					results <- diffPrepareResult{release: release, flags: flags, errors: []*ReleaseError{}}
+					results <- diffPrepareResult{release: release, flags: flags, errors: []*ReleaseError{}, files: files}
 				}
 			}
 		},
@@ -1153,6 +1175,13 @@ func (st *HelmState) DiffReleases(helm helmexec.Interface, additionalValues []st
 	}
 
 	preps, prepErrs := st.prepareDiffReleases(helm, additionalValues, workerLimit, detailedExitCode, includeTests, suppressSecrets, opts)
+
+	defer func() {
+		for _, p := range preps {
+			st.removeFiles(p.files)
+		}
+	}()
+
 	if len(prepErrs) > 0 {
 		return []ReleaseSpec{}, prepErrs
 	}
@@ -1288,21 +1317,6 @@ func (st *HelmState) TestReleases(helm helmexec.Interface, cleanup bool, timeout
 
 // Clean will remove any generated secrets
 func (st *HelmState) Clean() []error {
-	errs := []error{}
-
-	for _, release := range st.Releases {
-		for _, value := range release.generatedValues {
-			err := st.removeFile(value)
-			if err != nil {
-				errs = append(errs, err)
-			}
-		}
-	}
-
-	if len(errs) != 0 {
-		return errs
-	}
-
 	return nil
 }
 
@@ -1441,6 +1455,30 @@ func (st *HelmState) PrepareReleases(helm helmexec.Interface, helmfileCommand st
 	return nil
 }
 
+func (st *HelmState) TriggerGlobalPrepareEvent(helmfileCommand string) (bool, error) {
+	return st.triggerGlobalReleaseEvent("prepare", nil, helmfileCommand)
+}
+
+func (st *HelmState) TriggerGlobalCleanupEvent(helmfileCommand string) (bool, error) {
+	return st.triggerGlobalReleaseEvent("cleanup", nil, helmfileCommand)
+}
+
+func (st *HelmState) triggerGlobalReleaseEvent(evt string, evtErr error, helmfileCmd string) (bool, error) {
+	bus := &event.Bus{
+		Hooks:         st.Hooks,
+		StateFilePath: st.FilePath,
+		BasePath:      st.basePath,
+		Namespace:     st.OverrideNamespace,
+		Env:           st.Env,
+		Logger:        st.logger,
+		ReadFile:      st.readFile,
+	}
+	data := map[string]interface{}{
+		"HelmfileCommand": helmfileCmd,
+	}
+	return bus.Trigger(evt, evtErr, data)
+}
+
 func (st *HelmState) triggerPrepareEvent(r *ReleaseSpec, helmfileCommand string) (bool, error) {
 	return st.triggerReleaseEvent("prepare", nil, r, helmfileCommand)
 }
@@ -1641,7 +1679,7 @@ func (st *HelmState) timeoutFlags(helm helmexec.Interface, release *ReleaseSpec)
 	return flags
 }
 
-func (st *HelmState) flagsForUpgrade(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, error) {
+func (st *HelmState) flagsForUpgrade(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, []string, error) {
 	flags := []string{}
 	if release.Version != "" {
 		flags = append(flags, "--version", release.Version)
@@ -1683,7 +1721,7 @@ func (st *HelmState) flagsForUpgrade(helm helmexec.Interface, release *ReleaseSp
 			flags = append(flags, "--create-namespace")
 		} else if release.CreateNamespace != nil || st.HelmDefaults.CreateNamespace != nil {
 			// createNamespace was set explicitly, but not running supported version of helm - error
-			return nil, fmt.Errorf("releases[].createNamespace requires Helm 3.2.0 or greater")
+			return nil, nil, fmt.Errorf("releases[].createNamespace requires Helm 3.2.0 or greater")
 		}
 	}
 
@@ -1692,35 +1730,35 @@ func (st *HelmState) flagsForUpgrade(helm helmexec.Interface, release *ReleaseSp
 	var err error
 	flags, err = st.appendHelmXFlags(flags, release)
 	if err != nil {
-		return nil, err
+		return nil, nil, err
 	}
 
-	common, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
+	common, clean, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
 	if err != nil {
-		return nil, err
+		return nil, clean, err
 	}
-	return append(flags, common...), nil
+	return append(flags, common...), clean, nil
 }
 
-func (st *HelmState) flagsForTemplate(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, error) {
+func (st *HelmState) flagsForTemplate(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, []string, error) {
 	flags := []string{}
 
 	var err error
 	flags, err = st.appendHelmXFlags(flags, release)
 	if err != nil {
-		return nil, err
+		return nil, nil, err
 	}
 
 	flags = st.appendApiVersionsFlags(flags)
 
-	common, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
+	common, files, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
 	if err != nil {
-		return nil, err
+		return nil, files, err
 	}
-	return append(flags, common...), nil
+	return append(flags, common...), files, nil
 }
 
-func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, error) {
+func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, []string, error) {
 	flags := []string{}
 	if release.Version != "" {
 		flags = append(flags, "--version", release.Version)
@@ -1735,14 +1773,14 @@ func (st *HelmState) flagsForDiff(helm helmexec.Interface, release *ReleaseSpec,
 	var err error
 	flags, err = st.appendHelmXFlags(flags, release)
 	if err != nil {
-		return nil, err
+		return nil, nil, err
 	}
 
-	common, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
+	common, files, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
 	if err != nil {
-		return nil, err
+		return nil, files, err
 	}
-	return append(flags, common...), nil
+	return append(flags, common...), files, nil
 }
 
 func (st *HelmState) appendApiVersionsFlags(flags []string) []string {
@@ -1761,18 +1799,18 @@ func (st *HelmState) isDevelopment(release *ReleaseSpec) bool {
 	return result
 }
 
-func (st *HelmState) flagsForLint(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, error) {
-	flags, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
+func (st *HelmState) flagsForLint(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, []string, error) {
+	flags, files, err := st.namespaceAndValuesFlags(helm, release, workerIndex)
 	if err != nil {
-		return nil, err
+		return nil, files, err
 	}
 
 	flags, err = st.appendHelmXFlags(flags, release)
 	if err != nil {
-		return nil, err
+		return nil, files, err
 	}
 
-	return flags, nil
+	return flags, files, nil
 }
 
 func (st *HelmState) RenderValuesFileToBytes(path string) ([]byte, error) {
@@ -1845,6 +1883,16 @@ func (st *HelmState) ExpandedHelmfiles() ([]SubHelmfileSpec, error) {
 	return helmfiles, nil
 }
 
+func (st *HelmState) removeFiles(files []string) {
+	for _, f := range files {
+		if err := st.removeFile(f); err != nil {
+			st.logger.Warnf("Removing %s: %v", err)
+		} else {
+			st.logger.Debugf("Removed %s", f)
+		}
+	}
+}
+
 func (st *HelmState) generateTemporaryValuesFiles(values []interface{}, missingFileHandler *string) ([]string, error) {
 	generatedFiles := []string{}
 
@@ -1853,47 +1901,47 @@ func (st *HelmState) generateTemporaryValuesFiles(values []interface{}, missingF
 		case string:
 			paths, skip, err := st.storage().resolveFile(missingFileHandler, "values", typedValue)
 			if err != nil {
-				return nil, err
+				return generatedFiles, err
 			}
 			if skip {
 				continue
 			}
 
 			if len(paths) > 1 {
-				return nil, fmt.Errorf("glob patterns in release values and secrets is not supported yet. please submit a feature request if necessary")
+				return generatedFiles, fmt.Errorf("glob patterns in release values and secrets is not supported yet. please submit a feature request if necessary")
 			}
 			path := paths[0]
 
 			yamlBytes, err := st.RenderValuesFileToBytes(path)
 			if err != nil {
-				return nil, fmt.Errorf("failed to render values files \"%s\": %v", typedValue, err)
+				return generatedFiles, fmt.Errorf("failed to render values files \"%s\": %v", typedValue, err)
 			}
 
 			valfile, err := ioutil.TempFile("", "values")
 			if err != nil {
-				return nil, err
+				return generatedFiles, err
 			}
 			defer valfile.Close()
 
 			if _, err := valfile.Write(yamlBytes); err != nil {
-				return nil, fmt.Errorf("failed to write %s: %v", valfile.Name(), err)
+				return generatedFiles, fmt.Errorf("failed to write %s: %v", valfile.Name(), err)
 			}
 			st.logger.Debugf("successfully generated the value file at %s. produced:\n%s", path, string(yamlBytes))
 			generatedFiles = append(generatedFiles, valfile.Name())
 		case map[interface{}]interface{}, map[string]interface{}:
 			valfile, err := ioutil.TempFile("", "values")
 			if err != nil {
-				return nil, err
+				return generatedFiles, err
 			}
 			defer valfile.Close()
 			encoder := yaml.NewEncoder(valfile)
 			defer encoder.Close()
 			if err := encoder.Encode(typedValue); err != nil {
-				return nil, err
+				return generatedFiles, err
 			}
 			generatedFiles = append(generatedFiles, valfile.Name())
 		default:
-			return nil, fmt.Errorf("unexpected type of value: value=%v, type=%T", typedValue, typedValue)
+			return generatedFiles, fmt.Errorf("unexpected type of value: value=%v, type=%T", typedValue, typedValue)
 		}
 	}
 	return generatedFiles, nil
@@ -1926,8 +1974,6 @@ func (st *HelmState) generateVanillaValuesFiles(release *ReleaseSpec) ([]string,
 		return nil, err
 	}
 
-	release.generatedValues = append(release.generatedValues, generatedFiles...)
-
 	return generatedFiles, nil
 }
 
@@ -1957,8 +2003,6 @@ func (st *HelmState) generateSecretValuesFiles(helm helmexec.Interface, release
 		generatedFiles = append(generatedFiles, valfile)
 	}
 
-	release.generatedValues = append(release.generatedValues, generatedFiles...)
-
 	return generatedFiles, nil
 }
 
@@ -1978,17 +2022,21 @@ func (st *HelmState) generateValuesFiles(helm helmexec.Interface, release *Relea
 	return files, nil
 }
 
-func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, error) {
+func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *ReleaseSpec, workerIndex int) ([]string, []string, error) {
 	flags := []string{}
 	if release.Namespace != "" {
 		flags = append(flags, "--namespace", release.Namespace)
 	}
 
+	var files []string
+
 	generatedFiles, err := st.generateValuesFiles(helm, release, workerIndex)
 	if err != nil {
-		return nil, err
+		return nil, files, err
 	}
 
+	files = generatedFiles
+
 	for _, f := range generatedFiles {
 		flags = append(flags, "--values", f)
 	}
@@ -1998,7 +2046,7 @@ func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *R
 			if set.Value != "" {
 				renderedValue, err := renderValsSecrets(st.valsRuntime, set.Value)
 				if err != nil {
-					return nil, fmt.Errorf("Failed to render set value entry in %s for release %s: %v", st.FilePath, release.Name, err)
+					return nil, files, fmt.Errorf("Failed to render set value entry in %s for release %s: %v", st.FilePath, release.Name, err)
 				}
 				flags = append(flags, "--set", fmt.Sprintf("%s=%s", escape(set.Name), escape(renderedValue[0])))
 			} else if set.File != "" {
@@ -2006,7 +2054,7 @@ func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *R
 			} else if len(set.Values) > 0 {
 				renderedValues, err := renderValsSecrets(st.valsRuntime, set.Values...)
 				if err != nil {
-					return nil, fmt.Errorf("Failed to render set values entry in %s for release %s: %v", st.FilePath, release.Name, err)
+					return nil, files, fmt.Errorf("Failed to render set values entry in %s for release %s: %v", st.FilePath, release.Name, err)
 				}
 				items := make([]string, len(renderedValues))
 				for i, raw := range renderedValues {
@@ -2037,7 +2085,7 @@ func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *R
 		if len(envValErrs) != 0 {
 			joinedEnvVals := strings.Join(envValErrs, "\n")
 			errMsg := fmt.Sprintf("Environment Variables not found. Please make sure they are set and try again:\n%s", joinedEnvVals)
-			return nil, errors.New(errMsg)
+			return nil, files, errors.New(errMsg)
 		}
 		flags = append(flags, "--set", strings.Join(val, ","))
 	}
@@ -2045,7 +2093,7 @@ func (st *HelmState) namespaceAndValuesFlags(helm helmexec.Interface, release *R
 	 * END 'env' section for backwards compatibility
 	 **************/
 
-	return flags, nil
+	return flags, files, nil
 }
 
 // renderValsSecrets helper function which renders 'ref+.*' secrets
@@ -2145,7 +2193,7 @@ func (hf *SubHelmfileSpec) UnmarshalYAML(unmarshal func(interface{}) error) erro
 	return nil
 }
 
-func (st *HelmState) GenerateOutputDir(outputDir string, release ReleaseSpec) (string, error) {
+func (st *HelmState) GenerateOutputDir(outputDir string, release *ReleaseSpec) (string, error) {
 	// get absolute path of state file to generate a hash
 	// use this hash to write helm output in a specific directory by state file and release name
 	// ie. in a directory named stateFileName-stateFileHash-releaseName
diff --git a/pkg/tmpl/context_funcs.go b/pkg/tmpl/context_funcs.go
index fac7c8c..5d65e58 100644
--- a/pkg/tmpl/context_funcs.go
+++ b/pkg/tmpl/context_funcs.go
@@ -2,6 +2,7 @@ package tmpl
 
 import (
 	"fmt"
+	"github.com/roboll/helmfile/pkg/helmexec"
 	"golang.org/x/sync/errgroup"
 	"gopkg.in/yaml.v2"
 	"io"
@@ -97,25 +98,9 @@ func (c *Context) Exec(command string, args []interface{}, inputs ...string) (st
 	g.Go(func() error {
 		// We use CombinedOutput to produce helpful error messages
 		// See https://github.com/roboll/helmfile/issues/1158
-		bs, err := cmd.CombinedOutput()
+		bs, err := helmexec.Output(cmd)
 		if err != nil {
-			args := strings.Join(strArgs, ", ")
-			shownCmd := []string{command}
-			if len(args) > 0 {
-				shownCmd = append(shownCmd, args)
-			}
-
-			var out string
-
-			out += fmt.Sprintf("\n\nCOMMAND:\n%s", Indent(strings.Join(shownCmd, " "), "  "))
-
-			out += fmt.Sprintf("\n\nERROR:\n%s", Indent(err.Error(), "  "))
-
-			if len(bs) > 0 {
-				out += fmt.Sprintf("\n\nCOMBINED OUTPUT:\n%s", Indent(string(bs), "  "))
-			}
-
-			return fmt.Errorf("%v%s", err, out)
+			return err
 		}
 
 		bytes = bs
@@ -130,29 +115,6 @@ func (c *Context) Exec(command string, args []interface{}, inputs ...string) (st
 	return string(bytes), nil
 }
 
-// indents a block of text with an indent string
-func Indent(text, indent string) string {
-	var b strings.Builder
-
-	b.Grow(len(text) * 2)
-
-	lines := strings.Split(text, "\n")
-
-	last := len(lines) - 1
-
-	for i, j := range lines {
-		if i > 0 && i < last && j != "" {
-			b.WriteString("\n")
-		}
-
-		if j != "" {
-			b.WriteString(indent + j)
-		}
-	}
-
-	return b.String()
-}
-
 func (c *Context) ReadFile(filename string) (string, error) {
 	var path string
 	if filepath.IsAbs(filename) {


Test patch:
diff --git a/pkg/app/app_test.go b/pkg/app/app_test.go
index 4cb4614..884e7bc 100644
--- a/pkg/app/app_test.go
+++ b/pkg/app/app_test.go
@@ -4,6 +4,7 @@ import (
 	"bufio"
 	"bytes"
 	"fmt"
+	"github.com/roboll/helmfile/pkg/remote"
 	"io"
 	"log"
 	"os"
@@ -1505,11 +1506,13 @@ helmDefaults:
 		readFile:            testFs.ReadFile,
 		glob:                testFs.Glob,
 		abs:                 testFs.Abs,
+		directoryExistsAt:   testFs.DirectoryExistsAt,
 		fileExistsAt:        testFs.FileExistsAt,
 		fileExists:          testFs.FileExists,
 		Env:                 "default",
 		Logger:              helmexec.NewLogger(os.Stderr, "debug"),
 	}
+	app.remote = remote.NewRemote(app.Logger, "", app.readFile, app.directoryExistsAt, app.fileExistsAt)
 
 	expectNoCallsToHelm(app)
 
@@ -1597,6 +1600,7 @@ helmDefaults:
 		Env:                "default",
 		Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 	}
+	app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
 
 	expectNoCallsToHelm(app)
 
@@ -1675,6 +1679,7 @@ foo: FOO
 		Env:                "default",
 		Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 	}
+	app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
 
 	expectNoCallsToHelm(app)
 
@@ -1740,6 +1745,7 @@ foo: FOO
 		Env:                "default",
 		Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 	}
+	app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
 
 	expectNoCallsToHelm(app)
 
@@ -1823,6 +1829,7 @@ helmDefaults:
 		Env:                "test",
 		Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 	}
+	app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
 
 	expectNoCallsToHelm(app)
 
@@ -1897,6 +1904,7 @@ releases:
 		Env:                "default",
 		Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 	}
+	app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
 
 	expectNoCallsToHelm(app)
 
@@ -1955,6 +1963,7 @@ releases:
 		Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 	}
 
+	app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
 	expectNoCallsToHelm(app)
 
 	st, err := app.loadDesiredStateFromYaml(statePath, LoadOpts{Reverse: true})
@@ -2010,6 +2019,8 @@ releases:
 			Env:                "default",
 			Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 		}
+		app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
+
 		opts := LoadOpts{
 			CalleePath: statePath,
 			Environment: state.SubhelmfileEnvironmentSpec{
@@ -2122,6 +2133,7 @@ services:
 			Env:                "default",
 			Logger:             helmexec.NewLogger(os.Stderr, "debug"),
 		}
+		app.remote = remote.NewRemote(app.Logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
 
 		expectNoCallsToHelm(app)
 
diff --git a/pkg/app/diff_test.go b/pkg/app/diff_test.go
index 2069fdd..8b5127e 100644
--- a/pkg/app/diff_test.go
+++ b/pkg/app/diff_test.go
@@ -172,7 +172,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				// noop on frontend-v2
 				exectest.DiffKey{Name: "frontend-v2", Chart: "charts/frontend", Flags: "--kube-contextdefault--detailed-exitcode"}: nil,
@@ -389,7 +389,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -461,7 +461,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -483,7 +483,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -506,7 +506,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--namespacetestNamespace--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--namespacetestNamespace--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -529,7 +529,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--namespacetestNamespace--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--namespacetestNamespace--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -553,7 +553,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--namespacens2--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--namespacens1--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -577,7 +577,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--namespacens2--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--namespacens1--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -604,7 +604,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--tiller-namespacetns2--kube-contextdefault--namespacens2--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--tiller-namespacetns1--kube-contextdefault--namespacens1--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -630,7 +630,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--tiller-namespacetns2--kube-contextdefault--namespacens2--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--tiller-namespacetns1--kube-contextdefault--namespacens1--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -705,7 +705,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -737,7 +737,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -771,7 +771,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -803,7 +803,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -835,7 +835,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -867,7 +867,7 @@ releases:
 `,
 			},
 			detailedExitcode: true,
-			error:            "Identified at least on change",
+			error:            "Identified at least one change",
 			diffs: map[exectest.DiffKey]error{
 				exectest.DiffKey{Name: "bar", Chart: "mychart2", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
 				exectest.DiffKey{Name: "foo", Chart: "mychart1", Flags: "--kube-contextdefault--detailed-exitcode"}: helmexec.ExitError{Code: 2},
@@ -925,7 +925,7 @@ releases:
 			upgraded: []exectest.Release{},
 			// as we check for log output, set concurrency to 1 to avoid non-deterministic test result
 			concurrency: 1,
-			error:       "Identified at least on change",
+			error:       "Identified at least one change",
 			log: `processing file "helmfile.yaml" in directory "."
 first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{default map[] map[]}, overrode=<nil>
 first-pass uses: &{default map[] map[]}
diff --git a/pkg/app/two_pass_renderer_test.go b/pkg/app/two_pass_renderer_test.go
index d21450a..ce94535 100644
--- a/pkg/app/two_pass_renderer_test.go
+++ b/pkg/app/two_pass_renderer_test.go
@@ -1,6 +1,7 @@
 package app
 
 import (
+	"github.com/roboll/helmfile/pkg/remote"
 	"os"
 	"strings"
 	"testing"
@@ -14,6 +15,8 @@ import (
 
 func makeLoader(files map[string]string, env string) (*desiredStateLoader, *testhelper.TestFs) {
 	testfs := testhelper.NewTestFs(files)
+	logger := helmexec.NewLogger(os.Stdout, "debug")
+	r := remote.NewRemote(logger, testfs.Cwd, testfs.ReadFile, testfs.DirectoryExistsAt, testfs.FileExistsAt)
 	return &desiredStateLoader{
 		env:        env,
 		namespace:  "namespace",
@@ -22,6 +25,7 @@ func makeLoader(files map[string]string, env string) (*desiredStateLoader, *test
 		fileExists: testfs.FileExists,
 		abs:        testfs.Abs,
 		glob:       testfs.Glob,
+		remote:     r,
 	}, testfs
 }
 
diff --git a/pkg/state/create_test.go b/pkg/state/create_test.go
index b518de7..bc70a6a 100644
--- a/pkg/state/create_test.go
+++ b/pkg/state/create_test.go
@@ -1,7 +1,9 @@
 package state
 
 import (
+	"github.com/roboll/helmfile/pkg/remote"
 	"io/ioutil"
+	"os"
 	"path/filepath"
 	"reflect"
 	"testing"
@@ -18,7 +20,9 @@ func createFromYaml(content []byte, file string, env string, logger *zap.Sugared
 		logger:   logger,
 		readFile: ioutil.ReadFile,
 		abs:      filepath.Abs,
-		Strict:   true,
+
+		DeleteFile: os.Remove,
+		Strict:     true,
 	}
 	return c.ParseAndLoad(content, filepath.Dir(file), file, env, true, nil)
 }
@@ -108,7 +112,9 @@ bar: {{ readFile "bar.txt" }}
 	})
 	testFs.Cwd = "/example/path/to"
 
-	state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, nil, nil, "").ParseAndLoad(yamlContent, filepath.Dir(yamlFile), yamlFile, "production", true, nil)
+	r := remote.NewRemote(logger, testFs.Cwd, testFs.ReadFile, testFs.DirectoryExistsAt, testFs.FileExistsAt)
+	state, err := NewCreator(logger, testFs.ReadFile, testFs.FileExists, testFs.Abs, testFs.Glob, nil, nil, "", r).
+		ParseAndLoad(yamlContent, filepath.Dir(yamlFile), yamlFile, "production", true, nil)
 	if err != nil {
 		t.Fatalf("unexpected error: %v", err)
 	}
diff --git a/pkg/state/envvals_loader_test.go b/pkg/state/envvals_loader_test.go
index f4ed186..bb6feb2 100644
--- a/pkg/state/envvals_loader_test.go
+++ b/pkg/state/envvals_loader_test.go
@@ -2,6 +2,7 @@ package state
 
 import (
 	"github.com/google/go-cmp/cmp"
+	"github.com/roboll/helmfile/pkg/remote"
 	"go.uber.org/zap"
 	"io/ioutil"
 	"path/filepath"
@@ -23,7 +24,10 @@ func newLoader() *EnvironmentValuesLoader {
 		logger:   sugar,
 	}
 
-	return NewEnvironmentValuesLoader(storage, ioutil.ReadFile, sugar)
+	readFile := func(s string) ([]byte, error) { return []byte{}, nil }
+	dirExists := func(d string) bool { return false }
+	fileExists := func(f string) bool { return false }
+	return NewEnvironmentValuesLoader(storage, ioutil.ReadFile, sugar, remote.NewRemote(sugar, "/tmp", readFile, dirExists, fileExists))
 }
 
 // See https://github.com/roboll/helmfile/pull/1169
diff --git a/pkg/state/state_test.go b/pkg/state/state_test.go
index bcb5531..840d71e 100644
--- a/pkg/state/state_test.go
+++ b/pkg/state/state_test.go
@@ -705,7 +705,7 @@ func TestHelmState_flagsForUpgrade(t *testing.T) {
 				Version: tt.version,
 			}
 
-			args, err := state.flagsForUpgrade(helm, tt.release, 0)
+			args, _, err := state.flagsForUpgrade(helm, tt.release, 0)
 			if err != nil && tt.wantErr == "" {
 				t.Errorf("unexpected error flagsForUpgrade: %v", err)
 			}
diff --git a/test/diff-yamls.go b/test/diff-yamls.go
new file mode 100644
index 0000000..c53bb9e
--- /dev/null
+++ b/test/diff-yamls.go
@@ -0,0 +1,275 @@
+package main
+
+import (
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"github.com/go-test/deep"
+	"github.com/spf13/cobra"
+	"k8s.io/apimachinery/pkg/util/sets"
+	"k8s.io/apimachinery/pkg/util/yaml"
+)
+
+type diffSource string
+
+const (
+	diffSourceLeft  diffSource = "left"
+	diffSourceRight diffSource = "right"
+)
+
+var (
+	rootCmd = &cobra.Command{
+		Use:   "diff-yamls dir-with-yamls/ dir-with-yamls/",
+		Short: "Print any diff between the given directories",
+		Long: `Similar to the 'diff' command, but file contents
+are compared after being parsed as a set of Kubernetes manifests.`,
+		Args: cobra.ExactArgs(2),
+		Run:  run,
+	}
+)
+
+func run(cmd *cobra.Command, args []string) {
+	left := args[0]
+	right := args[1]
+	leftYamls, err := globYamlFilenames(left)
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "%v\n", err)
+		os.Exit(1)
+	}
+	rightYamls, err := globYamlFilenames(right)
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "%v\n", err)
+		os.Exit(1)
+	}
+
+	exitCode := 0
+
+	onlyInLeft := leftYamls.Difference(rightYamls)
+	if onlyInLeft.Len() > 0 {
+		exitCode = 1
+		for _, f := range onlyInLeft.List() {
+			fmt.Fprintf(os.Stderr, "Only in %s: %s\n", left, f)
+		}
+	}
+
+	onlyInRight := rightYamls.Difference(leftYamls)
+	if onlyInRight.Len() > 0 {
+		exitCode = 1
+		for _, f := range onlyInRight.List() {
+			fmt.Fprintf(os.Stderr, "Only in %s: %s\n", right, f)
+		}
+	}
+
+	inBoth := leftYamls.Intersection(rightYamls)
+	for _, f := range inBoth.List() {
+		leftPath := filepath.Join(left, f)
+		leftNodes, err := readManifest(leftPath)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "%v\n", err)
+			os.Exit(1)
+		}
+		rightPath := filepath.Join(right, f)
+		rightNodes, err := readManifest(rightPath)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "%v\n", err)
+			os.Exit(1)
+		}
+		ps := pairs{}
+		for _, node := range leftNodes {
+			if err := ps.add(node, diffSourceLeft); err != nil {
+				fmt.Fprintf(os.Stderr, "%v\n", err)
+				os.Exit(1)
+			}
+		}
+		for _, node := range rightNodes {
+			if err := ps.add(node, diffSourceRight); err != nil {
+				fmt.Fprintf(os.Stderr, "%v\n", err)
+				os.Exit(1)
+			}
+		}
+		for _, p := range ps.list {
+			switch {
+			case p.left != nil && p.right == nil:
+				fmt.Fprintf(os.Stderr, "Only in %s: %s\n", leftPath, p.left.getId())
+				exitCode = 1
+			case p.left == nil && p.right != nil:
+				fmt.Fprintf(os.Stderr, "Only in %s: %s\n", rightPath, p.right.getId())
+				exitCode = 1
+			default:
+				diff := deep.Equal(p.left, p.right)
+				if diff != nil {
+					id := p.left.getId()
+					fmt.Fprintf(os.Stderr, "< %s %s\n", id, leftPath)
+					fmt.Fprintf(os.Stderr, "> %s %s\n", id, rightPath)
+					for _, d := range diff {
+						fmt.Fprintf(os.Stderr, "%s\n", d)
+					}
+					exitCode = 1
+				}
+			}
+		}
+	}
+
+	os.Exit(exitCode)
+}
+
+func globYamlFilenames(dir string) (sets.String, error) {
+	matches, err := filepath.Glob(filepath.Join(dir, "*.yaml"))
+	if err != nil {
+		return nil, err
+	}
+	set := sets.String{}
+	for _, f := range matches {
+		set.Insert(filepath.Base(f))
+	}
+	return set, nil
+}
+
+type resource map[string]interface{}
+
+type meta struct {
+	apiVersion string
+	kind       string
+	name       string
+	namespace  string
+}
+
+func (res resource) getMeta() (meta, error) {
+	if len(res) == 0 {
+		return meta{}, nil
+	}
+	m := meta{}
+	apiVersion, _ := res["apiVersion"].(string)
+	m.apiVersion = apiVersion
+	kind, _ := res["kind"].(string)
+	m.kind = kind
+	metadata, _ := res["metadata"].(map[string]interface{})
+	name, _ := metadata["name"].(string)
+	m.name = name
+	namespace, _ := metadata["namespace"].(string)
+	m.namespace = namespace
+	return m, nil
+}
+
+func readManifest(path string) ([]resource, error) {
+	var err error
+	f, err := os.Open(path)
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+	decoder := yaml.NewYAMLToJSONDecoder(f)
+	resources := []resource{}
+	for {
+		r := make(resource)
+		err = decoder.Decode(&r)
+		if err != nil {
+			break
+		}
+		if len(r) > 0 {
+			resources = append(resources, r)
+		}
+	}
+	if err != nil && err != io.EOF {
+		return nil, err
+	}
+	return resources, nil
+}
+
+func (res resource) getId() string {
+	meta, err := res.getMeta()
+	if err != nil {
+		return fmt.Sprintf("%v", res)
+	}
+	ns := meta.namespace
+	if ns == "" {
+		ns = "~X"
+	}
+	nm := meta.name
+	if nm == "" {
+		nm = "~N"
+	}
+	gv := meta.apiVersion
+	if gv == "" {
+		gv = "~G_~V"
+	}
+	k := meta.kind
+	if k == "" {
+		k = "~K"
+	}
+	gvk := strings.Join([]string{gv, k}, "_")
+	return strings.Join([]string{gvk, ns, nm}, "|")
+
+}
+
+// lifted from kustomize/kyaml/kio/filters/merge3.go
+type pairs struct {
+	list []*pair
+}
+
+func (ps *pairs) isSameResource(meta1, meta2 meta) bool {
+	if meta1.name != meta2.name {
+		return false
+	}
+	if meta1.namespace != meta2.namespace {
+		return false
+	}
+	if meta1.apiVersion != meta2.apiVersion {
+		return false
+	}
+	if meta1.kind != meta2.kind {
+		return false
+	}
+	return true
+}
+
+func (ps *pairs) add(node resource, source diffSource) error {
+	nodeMeta, err := node.getMeta()
+	if err != nil {
+		return err
+	}
+	for i := range ps.list {
+		p := ps.list[i]
+		if ps.isSameResource(p.meta, nodeMeta) {
+			return p.add(node, source)
+		}
+	}
+	p := &pair{meta: nodeMeta}
+	if err := p.add(node, source); err != nil {
+		return err
+	}
+	ps.list = append(ps.list, p)
+	return nil
+}
+
+type pair struct {
+	meta  meta
+	left  resource
+	right resource
+}
+
+func (p *pair) add(node resource, source diffSource) error {
+	switch source {
+	case diffSourceLeft:
+		if p.left != nil {
+			return fmt.Errorf("left source already specified")
+		}
+		p.left = node
+	case diffSourceRight:
+		if p.right != nil {
+			return fmt.Errorf("right source already specified")
+		}
+		p.right = node
+	default:
+		return fmt.Errorf("unknown diff source value: %s", source)
+	}
+	return nil
+
+}
+
+func main() {
+	rootCmd.Execute()
+}
diff --git a/test/integration/charts/helmx/.helmignore b/test/integration/charts/helmx/.helmignore
new file mode 100644
index 0000000..f0c1319
--- /dev/null
+++ b/test/integration/charts/helmx/.helmignore
@@ -0,0 +1,21 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
diff --git a/test/integration/charts/helmx/Chart.yaml b/test/integration/charts/helmx/Chart.yaml
new file mode 100644
index 0000000..9b0b702
--- /dev/null
+++ b/test/integration/charts/helmx/Chart.yaml
@@ -0,0 +1,4 @@
+apiVersion: v1
+description: Chart for testing helmx features
+name: helmx
+version: 0.1.0
diff --git a/test/integration/charts/helmx/templates/_helpers.tpl b/test/integration/charts/helmx/templates/_helpers.tpl
new file mode 100644
index 0000000..d92e46b
--- /dev/null
+++ b/test/integration/charts/helmx/templates/_helpers.tpl
@@ -0,0 +1,16 @@
+{{/* vim: set filetype=mustache: */}}
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "helmx.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+*/}}
+{{- define "helmx.fullname" -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
diff --git a/test/integration/charts/helmx/templates/configmaps.yaml b/test/integration/charts/helmx/templates/configmaps.yaml
new file mode 100644
index 0000000..6cb00a9
--- /dev/null
+++ b/test/integration/charts/helmx/templates/configmaps.yaml
@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: release-name
+data:
+  name: {{ .Release.Name }}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: release-namespace
+data:
+  namespace: {{ .Release.Namespace }}
diff --git a/test/integration/charts/helmx/values.yaml b/test/integration/charts/helmx/values.yaml
new file mode 100644
index 0000000..e69de29
diff --git a/test/integration/happypath.yaml b/test/integration/happypath.yaml
index 8186ed8..03fea81 100644
--- a/test/integration/happypath.yaml
+++ b/test/integration/happypath.yaml
@@ -25,3 +25,17 @@ releases:
     values:
       - mysecret: {{ .Environment.Values.mysecret }}
       - values.yaml
+
+  - name: helmx
+    chart: ./charts/helmx
+    namespace: helmx-system
+    jsonPatches:
+      - target:
+          version: v1
+          kind: ConfigMap
+          name: release-name
+        patch:
+          - op: add
+            path: /metadata/annotations
+            value:
+              foo: bar
diff --git a/test/integration/run.sh b/test/integration/run.sh
index 14707b4..c1292a7 100755
--- a/test/integration/run.sh
+++ b/test/integration/run.sh
@@ -47,12 +47,15 @@ set -e
 info "Using namespace: ${test_ns}"
 # helm v2
 if helm version --client 2>/dev/null | grep '"v2\.'; then
+  helm_major_version=2
   info "Using Helm version: $(helm version --short --client | grep -o v.*$)"
   ${helm} init --wait --override spec.template.spec.automountServiceAccountToken=true
 # helm v3
 else
+  helm_major_version=3
   info "Using Helm version: $(helm version --short | grep -o v.*$)"
 fi
+info "Using Kustomize version: $(kustomize version --short | grep -o 'v[^ ]+')"
 ${helm} plugin install https://github.com/databus23/helm-diff --version v3.0.0-rc.7
 ${kubectl} get namespace ${test_ns} &> /dev/null && warn "Namespace ${test_ns} exists, from a previous test run?"
 $kubectl create namespace ${test_ns} || fail "Could not create namespace ${test_ns}"
@@ -73,9 +76,19 @@ info "Diffing ${dir}/happypath.yaml with limited context"
 bash -c "${helmfile} -f ${dir}/happypath.yaml diff --context 3 --detailed-exitcode; code="'$?'"; [ "'${code}'" -eq 2 ]" || fail "unexpected exit code returned by helmfile diff"
 
 info "Templating ${dir}/happypath.yaml"
-${helmfile} -f ${dir}/happypath.yaml template
+rm -rf ${dir}/tmp
+${helmfile} -f ${dir}/happypath.yaml --debug template --output-dir tmp
 code=$?
 [ ${code} -eq 0 ] || fail "unexpected exit code returned by helmfile template: ${code}"
+for output in $(ls -d ${dir}/tmp/*); do
+    # e.g. test/integration/tmp/happypath-877c0dd4-helmx/helmx
+    for release_dir in $(ls -d ${output}/*); do
+        release_name=$(basename ${release_dir})
+        golden_dir=${dir}/templates-golden/v${helm_major_version}/${release_name}
+        info "Comparing template output ${release_dir}/templates with ${golden_dir}"
+        ./diff-yamls ${golden_dir} ${release_dir}/templates || fail "unexpected diff in template result for ${release_name}"
+    done
+done
 
 info "Applying ${dir}/happypath.yaml"
 bash -c "${helmfile} -f ${dir}/happypath.yaml apply --detailed-exitcode; code="'$?'"; echo Code: "'$code'"; [ "'${code}'" -eq 2 ]" || fail "unexpected exit code returned by helmfile apply"
diff --git a/test/integration/templates-golden/v2/helmx/helmx.all.yaml b/test/integration/templates-golden/v2/helmx/helmx.all.yaml
new file mode 100644
index 0000000..6af9f07
--- /dev/null
+++ b/test/integration/templates-golden/v2/helmx/helmx.all.yaml
@@ -0,0 +1,17 @@
+---
+# Source: helmx/templates/helmx.all.yaml
+apiVersion: v1
+data:
+  namespace: helmx-system
+kind: ConfigMap
+metadata:
+  name: release-namespace
+---
+apiVersion: v1
+data:
+  name: helmx
+kind: ConfigMap
+metadata:
+  name: release-name
+  annotations:
+    foo: bar
diff --git a/test/integration/templates-golden/v2/httpbin/deployment.yaml b/test/integration/templates-golden/v2/httpbin/deployment.yaml
new file mode 100644
index 0000000..3493d17
--- /dev/null
+++ b/test/integration/templates-golden/v2/httpbin/deployment.yaml
@@ -0,0 +1,39 @@
+---
+# Source: httpbin/templates/deployment.yaml
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+metadata:
+  name: httpbin-httpbin
+  labels:
+    app: httpbin
+    chart: httpbin-0.1.0
+    release: httpbin
+    heritage: Tiller
+spec:
+  replicas: 1
+  strategy: {}
+  template:
+    metadata:
+      labels:
+        app: httpbin
+        release: httpbin
+    spec:
+      containers:
+      - name: httpbin
+        image: "docker.io/citizenstig/httpbin:latest"
+        imagePullPolicy: IfNotPresent
+        livenessProbe:
+          httpGet:
+            path: /
+            port: 8000
+        readinessProbe:
+          httpGet:
+            path: /
+            port: 8000        
+        ports:
+          - containerPort: 8000
+        resources:
+          {}
+
+status: {}
diff --git a/test/integration/templates-golden/v2/httpbin/service.yaml b/test/integration/templates-golden/v2/httpbin/service.yaml
new file mode 100644
index 0000000..0a0d86c
--- /dev/null
+++ b/test/integration/templates-golden/v2/httpbin/service.yaml
@@ -0,0 +1,21 @@
+---
+# Source: httpbin/templates/service.yaml
+apiVersion: v1
+kind: Service
+metadata:
+  name: httpbin-httpbin
+  labels:
+    app: httpbin
+    chart: httpbin-0.1.0
+    release: httpbin
+    heritage: Tiller
+spec:
+  type: LoadBalancer
+  ports:
+    - port: 8000
+      targetPort: 8000
+      protocol: TCP
+      name: httpbin
+  selector:
+    app: httpbin
+    release: httpbin
diff --git a/test/integration/templates-golden/v2/raw/resources.yaml b/test/integration/templates-golden/v2/raw/resources.yaml
new file mode 100644
index 0000000..cce0f8f
--- /dev/null
+++ b/test/integration/templates-golden/v2/raw/resources.yaml
@@ -0,0 +1,13 @@
+---
+# Source: raw/templates/resources.yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  labels:
+    app: raw
+    chart: raw-0.2.3
+    heritage: Tiller
+    release: raw
+  name: common-secret
+stringData:
+  mykey: MYSECRET
diff --git a/test/integration/templates-golden/v3/helmx/helmx.all.yaml b/test/integration/templates-golden/v3/helmx/helmx.all.yaml
new file mode 100644
index 0000000..f036ace
--- /dev/null
+++ b/test/integration/templates-golden/v3/helmx/helmx.all.yaml
@@ -0,0 +1,18 @@
+---
+# Source: helmx/templates/helmx.all.yaml
+apiVersion: v1
+data:
+  namespace: helmx-system
+kind: ConfigMap
+metadata:
+  name: release-namespace
+---
+# Source: helmx/templates/helmx.all.yaml
+apiVersion: v1
+data:
+  name: helmx
+kind: ConfigMap
+metadata:
+  name: release-name
+  annotations:
+    foo: bar
diff --git a/test/integration/templates-golden/v3/httpbin/deployment.yaml b/test/integration/templates-golden/v3/httpbin/deployment.yaml
new file mode 100644
index 0000000..97ff6a0
--- /dev/null
+++ b/test/integration/templates-golden/v3/httpbin/deployment.yaml
@@ -0,0 +1,39 @@
+---
+# Source: httpbin/templates/deployment.yaml
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+metadata:
+  name: httpbin-httpbin
+  labels:
+    app: httpbin
+    chart: httpbin-0.1.0
+    release: httpbin
+    heritage: Helm  
+spec:
+  replicas: 1
+  strategy: {}
+  template:
+    metadata:
+      labels:
+        app: httpbin
+        release: httpbin
+    spec:
+      containers:
+      - name: httpbin
+        image: "docker.io/citizenstig/httpbin:latest"
+        imagePullPolicy: IfNotPresent
+        livenessProbe:
+          httpGet:
+            path: /
+            port: 8000
+        readinessProbe:
+          httpGet:
+            path: /
+            port: 8000        
+        ports:
+          - containerPort: 8000
+        resources:
+          {}
+
+status: {}
diff --git a/test/integration/templates-golden/v3/httpbin/service.yaml b/test/integration/templates-golden/v3/httpbin/service.yaml
new file mode 100644
index 0000000..2b0e720
--- /dev/null
+++ b/test/integration/templates-golden/v3/httpbin/service.yaml
@@ -0,0 +1,21 @@
+---
+# Source: httpbin/templates/service.yaml
+apiVersion: v1
+kind: Service
+metadata:
+  name: httpbin-httpbin
+  labels:
+    app: httpbin
+    chart: httpbin-0.1.0
+    release: httpbin
+    heritage: Helm
+spec:
+  type: LoadBalancer
+  ports:
+    - port: 8000
+      targetPort: 8000
+      protocol: TCP
+      name: httpbin
+  selector:
+    app: httpbin
+    release: httpbin
diff --git a/test/integration/templates-golden/v3/raw/resources.yaml b/test/integration/templates-golden/v3/raw/resources.yaml
new file mode 100644
index 0000000..233c55d
--- /dev/null
+++ b/test/integration/templates-golden/v3/raw/resources.yaml
@@ -0,0 +1,13 @@
+---
+# Source: raw/templates/resources.yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  labels:
+    app: raw
+    chart: raw-0.2.3
+    heritage: Helm
+    release: raw
+  name: common-secret
+stringData:
+  mykey: MYSECRET