Repository: GoogleContainerTools/kpt
Pull request #2600 (2021-11-29 01:11:09)

changed default image pull policy to if-not-present
`kpt fn` commands support command flag called `image-pull-policy` that determines image fetch behavior for KRM functions. Current default value is `AlwaysPull` which means `kpt` will always try to pull the image using `docker pull` even if the function image is present locally. In addition, current code also calls `docker inspect` to check if the image is present locally for `ifnotpresent`.

This PR does the following:
 - Changes the default value for `image-pull-policy` to `if-not-present` i.e. pull image only if it doesn't exists locally.
 - Removes the `docker pull` and `docker image inspect` calls instead relays the `image-pull-policy` flag to `docker run` command as is. This will eliminate two docker calls and thus, make the `fn render` and `fn eval` faster.
 - To support the above implementation, we had to filter out docker CLI command  (especially progress messages). This was needed because output from executing KRM function and `docker run` will be emitted on same `stderr` output stream.

 

sdowell: Code changes LGTM

Patch:
diff --git a/Makefile b/Makefile
index bb94d116..b9f4d028 100644
--- a/Makefile
+++ b/Makefile
@@ -87,7 +87,7 @@ test-fn-eval: build
 
 # target to run e2e tests for "kpt fn eval" command
 test-live-apply: build
-	PATH=$(GOBIN):$(PATH) go test -v -timeout=20m --tags=kind --run=TestLiveApply/testdata/live-apply/$(T)  ./e2e/
+	PATH=$(GOBIN):$(PATH) go test -v -timeout=20m --tags=kind -p 2 --run=TestLiveApply/testdata/live-apply/$(T)  ./e2e/
 
 vet:
 	go vet ./...
diff --git a/docs/design-docs/01-oci-support.md b/docs/design-docs/01-oci-support.md
new file mode 100644
index 00000000..11d925d0
--- /dev/null
+++ b/docs/design-docs/01-oci-support.md
@@ -0,0 +1,474 @@
+# OCI Support
+
+* Author(s): Louis Dejardin, @loudej
+* Approver: \<kpt-maintainer\>
+
+>    Every feature will need design sign off an PR approval from a core
+>    maintainer.  If you have not got in touch with anyone yet, you can leave
+>    this blank and we will try to line someone up for you.
+
+## Why
+
+Systems that deal with packaging software or bundling configuration often have
+an atomic, versionable artifact. This artifact can exist as a source of
+truth independant from the source controlled content from which it was built.
+
+It is also very common for those packages to have an associated feed or repository
+which can receive those packages as they are published, and make them available for 
+download as needed. In many companies, using `git` as the as the repository and source
+of truth for production configuration comes with challenges.
+
+This design document proposes to add `OCI` as an alternative to `git` for publishing and
+distributing Kpt config packages. As a packaging format, it is well understood and documented.
+As a repository format, it leverages existing container registries for pushing and pulling
+config as image content. For security and production configuration management, if a 
+company has practices for managing Docker container images in private registries, then
+the same practices and security model can be applied to config package images in private
+registries.
+
+https://github.com/GoogleContainerTools/kpt/issues/2300
+
+## Design
+
+### Design Assumptions
+
+The first stage of OCI support comes from adding support for `oci` in the places
+where `git` appears today.
+
+An image tag is used in the same way a git branch or tag would be used.
+
+An image digest is used in the same way a git commit would be used.
+
+The scope of a single image is one root package, with any number of optional sub-packages.
+
+The structure of the image is a single tar layer. The root `Kptfile` is in the base directory from the tar layer's point of view. It contains only the Kpt package files, no entrypoint or executables.
+
+A package image should not be confused with a container image. Container images are executable by software, like `Docker`, and package images are purely configuration data.
+
+### Config chages
+
+The `Kptfile` structures for `upstream` and `upstreamLock` have `oci` in addition to `git` properties. The `type` property also has the string `oci` added as an accepted value.
+
+```yaml
+upstream:
+  type: oci
+  oci:
+    image: 'IMAGE:TAG'
+upstreamLock:
+  type: oci
+  oci:
+    image: 'IMAGE:DIGEST'
+```
+
+New verions of `kpt` will support existing `Kptfile`. The file structures and `git` functionality is unchanged.
+
+Existing versions of `kpt` will support `Kptfile` with `upstream` based on `git` for the same reason. The structure and meaning of existing fields is not changed.
+
+Existing versions of `kpt` will not support `Kptfile` with `upstream` based on `oci`. The `type` value, and missing `git` information will fail validation. The `kpt` binary used will need to be upgraded.
+
+### Command changes
+
+### `kpt pkg get`
+
+The argument that determines upstream today is parsed into `repo`, `ref`, and `path`, and is implicitly a `git` location.
+
+To support `oci`, it will be necessary to extract different values in a way that's unambiguous. Unfortunately, OCI image names have no Uri prefix, and are indistinguishable from a valid path or file name.
+
+To solve this, using [Helm](https://helm.sh/docs/topics/registries/#other-subcommands) as an example, the prefix `oci://` can be used. This ensures that selecting `oci` protocol isn't accidental, and it won't collide with other location formats that may be added.
+
+```shell
+# clone package as new folder
+kpt pkg get oci://us-docker.pkg.dev/the-project-id/the-repo-name/the-package:v3 my-package
+```
+
+Because OCI image reference already has a convention for `image:tag` references, using `:v3` should be used instead of `@v3` for version. It will be more intuitive how it relates to the registry, and easier to cut and paste values.
+
+### `kpt pkg get` sub-packages
+
+It is possible to use `kpt pkg get` to add sub-packages to a target location.
+
+Syntax for a sub-package target location is unchanged, it's a normal filesystem path.
+
+Syntax for an OCI sub-package source location requires the ability to tell when an image name ends and a sub-package path inside that image begins. In `git` this requires an explicit `.git` extension at the transition, and in `.oci` this requires double slash.
+
+```shell
+# clone sub-package as new sub-folder
+kpt pkg get oci://us-docker.pkg.dev/the-project-id/the-repo-name/the-package//simple/example:v3 my-package/simple/my-example
+```
+
+### `kpt pkg update`
+
+The command for update is not changed, but when the `upstream` is `oci` then the `@VERSION` is used to change the `upstream` image's `tag` or `digest` value.
+
+To update to an image tag, `kpt pkg update @v14` and `kpt pkg update DIR@v14` will assign the `:v14` tag onto the upstream image.
+
+```yaml
+upstream:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/the-project-id/the-repo-name/the-package:v14
+```
+
+To update to an upstream digest, `kpt pkg update @sha256:{SHA256_HEX}` and `kpt pkg update DIR@sha256:{SHA256_HEX}` will assign `@sha256:{SHA256_HEX}` as the new upstream image digest.
+
+```yaml
+upstream:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/the-project-id/the-repo-name/the-package@sha256:8815143a333cb9d2cb341f10b984b22f3b8a99fe
+```
+
+Calling `kpt pkg update` and `kpt pkg update DIR` will perform an update without changing the upstream image name.
+
+At that point, if the `upstream` is an `image:tag` that is to discover the current `image:digest` for tag, otherwise the `upstream` value for `image:digest` is used. In either case, the `upstreamLock` is changed to point at that new `image:digest`. 
+
+The package contents of the old and new `upstreamLock` image digest are fetched to temp folders, and are the basis of the 3-way merge to update the target package.
+
+### `kpt pkg diff`
+
+The `kpt pkg diff` command is identical to `kpt pkg update` in the way that `[PKG_PATH@VERSION]` argument is mapped to OCI concepts.
+
+### Command additions
+
+Although it is possible to create and push an OCI image using a combination of commands like `tar` and `gcrane`, that
+doesn't provide a very complete end to end experience. Because kpt would already be built with the same OCI go module used 
+by `gcrane`, it is not difficult to support additional commands to move pull and push package contents from local folders
+to remote images and back.
+
+### ` kpt pkg pull`
+
+```
+Usage: kpt pkg pull oci://{IMAGE[:TAG|@sha256:DIGEST]} [DIR] 
+  DIR                         Destination folder for image contents. Default folder name is the last part of the IMAGE path.
+  IMAGE[:TAG|@sha256:DIGEST]  Name of image to pull contents from, with optional TAG or DIGEST. Default TAG is `Latest`
+```
+
+This command is the reverse of push. An image can be pulled from a repository to a local folder, modified, and pushed
+back to the same location, same location with different TAG, or entirely different location.
+
+The target DIR is optional, following the conventions of `kpg pkg get`, and will default to the final image/path segment.
+
+`kpt pkg pull` works on git uri as well. This may be used, for example, to mirror a set of known blueprints into a private 
+oci registry.
+
+### ` kpt pkg push`
+
+```
+Usage: kpt pkg push [DIR@VERSION] [--origin oci://{IMAGE[:TAG]}] [--increment]
+  DIR@VERSION     Folder containing package root Kptfile. Default is current directory.
+                  Optional @VERSION changes tag or branch to push onto. Default is most recently pulled or pushed tag.
+  --origin        Name of image to push contents onto, with optional TAG to assign to resulting commit.
+                  Default is to use most recently pulled/pushed image. Required if Kptfile does not have an origin.
+  --increment     Increase the version by 1 while pushing. Default is to leave the origin's TAG or DIR@VERSION unchanged.
+                  The Kptfile's image TAG is also updated to the new value. 
+```
+
+This command will `tar` the contents of the package into a single image layer, and push it into the OCI repository. For
+Google Artifact Registry and Google Container Registry, the current `gcloud auth` SSO credentials are used.
+
+The simplest form of the command is `kpt pkg push` or `kpt pkg push DIR` which will push the current contents back to
+the IMAGE:TAG location that was saved when `kpt pkg pull` was run.
+
+The synxax `kpt pkg push @VERSION` or `kpt pkg push DIR@VERSION` will push back to the image location it came from, but with a new TAG name or version. Examples are `kpt pkt push @draft` or `kpt pkg push @v4`
+
+If the Kptfile was not obtained by `kpt pkg pull` - for example it's a new package from `kpt pkg init` or `kpt pkg get` - then
+the first `kpt pkg push` will require an `--origin IMAGE:TAG` option to provide the target location. It is only necessary on the first
+call.
+
+Finally, if the IMAGE's TAG value is a valid version number, the `--increment` switch can be used to add 1 to the current value before pushing.
+
+In the simplest case a `v1` is changed to `v2`, and `1` is changed to `2`, but any TAG that is a valid semver (with optional leading 'v') will have the smallest part of the number incremented. So `v1.0` becomes `v1.1`, `v1.0.0` becomes `v1.0.1`, and `v4.1.9-alpha` becomes `v4.1.10-alpha`
+
+#### Comparison of `pkg get` and `pkg pull`
+
+Starting with a simple root package, and an orange variant with root as the upstream:
+
+```
+-- root
+  \-- orange {upstream: root}
+```
+
+The purpose of `kpt get` is to create a new leaf node. This is done by creating the initial copy of the new leaf package in a
+local folder. This has the side-effects of altering the kptfile name, the upstream values to point at the source, and makes appropriate 
+changes to sub-package metadata.
+
+As an example, after running `kpt pkg get scheme://repo/root green` and `kpt pkg get scheme://repo/orange blue` the `green` and 
+`blue` local folder packages are appended to the inheritance tree like this:
+
+```
+-- root
+  \-- orange {upstream: root}
+  | `-- blue {upstream: orange}  ** working copy in ./blue **
+  \-- green {upstream: root}     ** working copy in ./green **
+```
+
+By comparison, `kpt pkg pull` does not create a new package node or identity - it only extracts a copy of existing package
+contents to a working directory. In this example, if the user additionally ran `kpt pkg pull scheme://repo/root root` and 
+`kpt pkg pull scheme://repo/orange orange` the overall state would be this:
+
+```
+-- root                          ** working copy in ./root **
+  \-- orange {upstream: root}    ** working copy in ./orange **
+  | `-- blue {upstream: orange}  ** working copy in ./blue **
+  \-- green {upstream: root}     ** working copy in ./green **
+```
+
+### Alternatives to push/pull
+
+There are several ways that pull and push could appear as commands. Those two names are very conventional, but 
+alternatives to consider could be:
+
+### `kpt pkg copy`
+
+```
+Usage: kpt pkg copy {SOURCE} {DEST}
+  SOURCE  Package source location: a local DIR, or `oci://` image, or git repo and path
+  DEST    Package destination: a local DIR, or `oci://` image.
+```
+
+Puts a copy of the SOURCE package at the DEST location. The package contents would be entirely unchanged by this operation (unlike `kpt pkg get`).
+
+To pull from remote image to local folder:
+
+```
+kpt pkg copy \
+  oci://us-docker.pkg.dev/the-project-id/the-repo-name/the-package:v14 \
+  the-package
+```
+
+To push from local folder to new remote image tag:
+
+```
+kpt pkg copy \
+  the-package \
+  oci://us-docker.pkg.dev/the-project-id/the-repo-name/the-package:v15
+```
+ 
+To copy a package image from one OCI repo to another:
+
+```
+kpt pkg copy \
+  oci://us-docker.pkg.dev/the-project-id/dev-blueprints/the-package:v25 \
+  oci://us-docker.pkg.dev/the-project-id/prod-blueprints/the-package:v25
+```
+
+To copy a package from a git location to an OCI repo:
+
+```
+kpt pkg copy \
+  https://github.com/GoogleCloudPlatform/blueprints.git/catalog/gke@main \
+  oci://us-docker.pkg.dev/the-project-id/gcp-catalog/gke:latest
+```
+
+## User Guide
+
+### Creating a package respository
+
+Before kpt packages can be pushed and pulled as OCI images, a suitable repository 
+must be created. Google Artifact Registry and Google Container Registry are both
+excellent choices.
+
+```shell
+# Choose names and locations
+LOCATION="us"
+PROJECT_ID="kpt-demo-73823"
+REPOSITORY_NAME="blueprints"
+
+# Base name for any images in this repository
+REPOSITORY="${LOCATION}-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY_NAME}"
+
+# Create the repository
+gcloud artifacts repositories create --location="${LOCATION}" --repository-format=docker --project="${PROJECT_ID}" "${REPOSITORY_NAME}"
+```
+
+### Creating and pushing a new package
+
+Creating a new package is no different. But when ready to publish, the `kpt pkg push` command 
+is used instead of source control operations.
+
+```shell
+# A package in a new directory
+mkdir hello-world
+kpt pkg init hello-world --description="A simple blueprint"
+
+# Store the contents in the repository, tagged as v1
+kpt pkg push hello-world --image=${REPOSITORY}/hello-world:v1
+
+# The local files are not needed any more, pushing has stored them all
+rm -r hello-world
+```
+
+### Pulling and updating a package
+
+Because the package folder was discarded earlier, the `kpt pkg pull` command 
+is used to place the contents of a particular version at a location. These set of
+commands may be run in `Cloud Build` steps as well, if you are automating the
+publication of packages as part of a CI/CD process.
+
+```shell
+# Recreate the folder and extract the pulled image
+kpt pkg pull hello-world --image=${REPOSITORY}/hello-world:v1
+
+# Add a sub-package from a git repo
+kpt pkg get https://github.com/GoogleCloudPlatform/blueprints.git/catalog/bucket hello-world/my-bucket
+
+# Render to be sure contents are hydrated, and push to a new version tag
+kpt pkg render hello-world
+kpt pkg push hello-world --image=${REPOSITORY}/hello-world:v2
+```
+
+Similar to container image tags, the package image tags like `:v1` and `:v2` above may be used any
+number of ways based on your preferred workflows. The tag `:latest` is used by default if all
+pulls and pushes should read from and overwrite the same location. Semantic tags like `:draft` and
+environmental tags like `:dev`, `:qa`, and `:prod` may be also used.
+
+No matter what tags are used, the image repository and `kpt` cli will treat them as an alphanumeric 
+label.
+
+### Using OCI repository as an upstream
+
+In addition to providing storage for packages, an OCI registry may also 
+be used as a source of upstream images to clone. The `oci://` prefix on this
+command is required to ensure 
+
+```shell
+# Clone the hello-world v1 blueprint into a new folder
+kpt pkg get oci://${REPOSITORY}/hello-world:v1 greetings-planet
+
+# Push the results to the repository, using default `latest` tag in this example
+kpt pkg push greetings-planet --image=${REPOSITORY}/greetings-planet
+```
+
+Looking in the `greetings-planet/Kptfile` at this point will show that
+the `hello-world:v1` image is the `upstream`, and the `upstreamLock` will show
+exactly the digest that this clone is up-to-date with. 
+
+```yaml
+apiVersion: kpt.dev/v1
+kind: Kptfile
+metadata:
+  name: greetings-planet
+upstream:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/kpt-demo-73823/blueprints/hello-world:v1
+  updateStrategy: resource-merge
+upstreamLock:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/kpt-demo-73823/blueprints/hello-world@sha256:1632e00af3fe858c5e3b3f9e75c16e6327449155
+```
+
+### Adding a subpackage from an OCI upstream subfolder
+
+Often a folder inside a package is meant to be used as a way to create "more of the same".
+
+To use an OCI image subfolder as the source of a subpackage, the path is added in a 
+way that's distinct from the image itself.
+
+```shell
+# Clone the hello-world v1 blueprint into a new folder
+kpt pkg get oci://${REPOSITORY}/hello-world//my-bucket:v1 greetings-planet/another-bucket
+```
+
+The `greetings-planet` package will now contain both a `greetings-planet/my-bucket` as well as a
+`greetings-planet/another-bucket` folder. The contents in locations will now both receive changes
+when the upstream `hello-world/my-bucket` is updated.
+
+### Updating package with upstream changes
+
+The value of the upstream image tag is used to `kpt pkg update` to a specific version.
+This works no matter if the tag appears to look like a version number or not.
+
+```shell
+# Update the greetings-planet by applying any differences between the upstreamLock digest and the `v2` tag
+kpt pkg update greetings-planet@v2
+
+# Overwrite the `greetings-planet:latest` image with the folder contents
+kpt pkg push greetings-planet --image=${REPOSITORY}/greetings-planet
+```
+
+The Kptfile will now show that the `upstream` and `upstreamLock` have both been changed.
+
+```yaml
+apiVersion: kpt.dev/v1
+kind: Kptfile
+metadata:
+  name: greetings-planet
+upstream:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/kpt-demo-73823/blueprints/hello-world:v2
+  updateStrategy: resource-merge
+upstreamLock:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/kpt-demo-73823/blueprints/hello-world@sha256:a6f1ed69c6ab51e2a148f6d4926bccb24c843887
+```
+
+Just like with a git upstream, it is also possible to `kpt pkg update` without providing a different
+tag or version value. This is similar to pulling from a remote branch where the name of the branch does 
+change but the latest commit on that branch is a different hash.
+
+In that case the Kptfile `upstream` tag will not change, but if that tag has been overwritten with
+new contents then the differences between `upstreamLock` and current contents will be applied to the local copy,
+and the `upstreamLock` will be changed tot he current digest.
+
+### Updating package to a specific upstream push
+
+Much like you can `kpt pkg update` to a specific commit hash in git, you can update to an
+exact image digest with OCI. This is an even more precise reference than by a version number because,
+like git commits, the image digest is based on the package file contents and cannot be forged or altered.
+
+```shell
+# Update instread to an exact image digest of the upstream location
+kpt pkg update greetings-planet@sha256:3b42daa41102fa83bce07bd82a72edcd691868d6
+```
+
+The resulting Kptfile in the local folder will look like this.
+
+```yaml
+apiVersion: kpt.dev/v1
+kind: Kptfile
+metadata:
+  name: greetings-planet
+upstream:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/kpt-demo-73823/blueprints/hello-world@sha256:3b42daa41102fa83bce07bd82a72edcd691868d6
+  updateStrategy: resource-merge
+upstreamLock:
+  type: oci
+  oci:
+    image: us-docker.pkg.dev/kpt-demo-73823/blueprints/hello-world@sha256:3b42daa41102fa83bce07bd82a72edcd691868d6
+```
+
+## Open Issues/Questions
+
+> Please list any open questions here in the following format:
+> 
+> ### \<Question\>
+> 
+> Resolution: Please list the resolution if resolved during the design process or
+> specify __Not Yet Resolved__
+
+### What additional container registries should be supported?
+
+The protocol and information is the same. It would mainly be a question
+of how the credentials for the call are provided.
+
+### What commands on exising kpt binary will work on Kptfile with `oci`
+
+It may be possible `kpt` commands that to not process `upstream` structures
+may not require update to work correctly. `kpt fn` and `kpt live` commands
+should be tested to see how they behave.
+
+## Alternatives Considered
+
+If there is an industry precedent or alternative approaches please list them 
+here as well as citing *why* you decided not to pursue those paths.
+
+### \<Approach\>
+
+Links and description of the approach, the pros and cons identified during the 
+design. 
diff --git a/go.sum b/go.sum
index b34fb99e..90451494 100644
--- a/go.sum
+++ b/go.sum
@@ -1037,8 +1037,6 @@ sigs.k8s.io/kustomize/cmd/config v0.9.13/go.mod h1:7547FLF8W/lTaDf0BDqFTbZxM9zqw
 sigs.k8s.io/kustomize/kustomize/v4 v4.2.0/go.mod h1:MOkR6fmhwG7hEDRXBYELTi5GSFcLwfqwzTRHW3kv5go=
 sigs.k8s.io/kustomize/kyaml v0.11.0/go.mod h1:GNMwjim4Ypgp/MueD3zXHLRJEjz7RvtPae0AwlvEMFM=
 sigs.k8s.io/kustomize/kyaml v0.12.0/go.mod h1:FTJxEZ86ScK184NpGSAQcfEqee0nul8oLCK30D47m4E=
-sigs.k8s.io/kustomize/kyaml v0.13.1-0.20211122190718-01420768c8ef h1:+S4oQAuqVk4Nmy28fALpv93AgW0UKQWjmqUwSNmIWpE=
-sigs.k8s.io/kustomize/kyaml v0.13.1-0.20211122190718-01420768c8ef/go.mod h1:FTJxEZ86ScK184NpGSAQcfEqee0nul8oLCK30D47m4E=
 sigs.k8s.io/kustomize/kyaml v0.13.1-0.20211203194734-cd2c6a1ad117 h1:4ePwpEnROoPqjaY6WC98EiRpYrmJjDHV6KA7PGDmpdQ=
 sigs.k8s.io/kustomize/kyaml v0.13.1-0.20211203194734-cd2c6a1ad117/go.mod h1:FTJxEZ86ScK184NpGSAQcfEqee0nul8oLCK30D47m4E=
 sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
diff --git a/internal/cmdrender/cmdrender.go b/internal/cmdrender/cmdrender.go
index ee5bca1d..2683a6f2 100644
--- a/internal/cmdrender/cmdrender.go
+++ b/internal/cmdrender/cmdrender.go
@@ -45,7 +45,7 @@ func NewRunner(ctx context.Context, parent string) *Runner {
 		"path to a directory to save function results")
 	c.Flags().StringVarP(&r.dest, "output", "o", "",
 		fmt.Sprintf("output resources are written to provided location. Allowed values: %s|%s|<OUT_DIR_PATH>", cmdutil.Stdout, cmdutil.Unwrap))
-	c.Flags().StringVar(&r.imagePullPolicy, "image-pull-policy", string(fnruntime.AlwaysPull),
+	c.Flags().StringVar(&r.imagePullPolicy, "image-pull-policy", string(fnruntime.IfNotPresentPull),
 		fmt.Sprintf("pull image before running the container. It must be one of %s, %s and %s.", fnruntime.AlwaysPull, fnruntime.IfNotPresentPull, fnruntime.NeverPull))
 	cmdutil.FixDocs("kpt", parent, c)
 	r.Command = c
diff --git a/internal/docs/generated/fndocs/docs.go b/internal/docs/generated/fndocs/docs.go
index 210f7413..f37a533c 100644
--- a/internal/docs/generated/fndocs/docs.go
+++ b/internal/docs/generated/fndocs/docs.go
@@ -90,7 +90,7 @@ Flags:
     Container image of the function to execute e.g. ` + "`" + `gcr.io/kpt-fn/set-namespace:v0.1` + "`" + `.
     For convenience, if full image path is not specified, ` + "`" + `gcr.io/kpt-fn/` + "`" + ` is added as default prefix.
     e.g. instead of passing ` + "`" + `gcr.io/kpt-fn/set-namespace:v0.1` + "`" + ` you can pass ` + "`" + `set-namespace:v0.1` + "`" + `.
-    ` + "`" + `eval` + "`" + ` executes only one function, so do not use ` + "`" + `--exec-path` + "`" + ` flag with this flag.
+    ` + "`" + `eval` + "`" + ` executes only one function, so do not use ` + "`" + `--exec` + "`" + ` flag with this flag.
   
   --image-pull-policy:
     If the image should be pulled before rendering the package(s). It can be set
diff --git a/internal/fnruntime/container.go b/internal/fnruntime/container.go
index 4eca4358..61132ea2 100644
--- a/internal/fnruntime/container.go
+++ b/internal/fnruntime/container.go
@@ -15,6 +15,7 @@
 package fnruntime
 
 import (
+	"bufio"
 	"bytes"
 	"context"
 	goerrors "errors"
@@ -34,11 +35,10 @@ import (
 type containerNetworkName string
 
 const (
-	networkNameNone     containerNetworkName = "none"
-	networkNameHost     containerNetworkName = "host"
-	defaultLongTimeout  time.Duration        = 5 * time.Minute
-	defaultShortTimeout time.Duration        = 5 * time.Second
-	dockerBin           string               = "docker"
+	networkNameNone    containerNetworkName = "none"
+	networkNameHost    containerNetworkName = "host"
+	defaultLongTimeout time.Duration        = 5 * time.Minute
+	dockerBin          string               = "docker"
 
 	AlwaysPull       ImagePullPolicy = "Always"
 	IfNotPresentPull ImagePullPolicy = "IfNotPresent"
@@ -85,13 +85,6 @@ type ContainerFn struct {
 // It reads the input from the given reader and writes the output
 // to the provided writer.
 func (f *ContainerFn) Run(reader io.Reader, writer io.Writer) error {
-	// check and pull image before running to avoid polluting CLI
-	// output
-	err := f.prepareImage()
-	if err != nil {
-		return err
-	}
-
 	errSink := bytes.Buffer{}
 	cmd, cancel := f.getDockerCmd()
 	defer cancel()
@@ -105,7 +98,7 @@ func (f *ContainerFn) Run(reader io.Reader, writer io.Writer) error {
 			return &ExecError{
 				OriginalErr:    exitErr,
 				ExitCode:       exitErr.ExitCode(),
-				Stderr:         errSink.String(),
+				Stderr:         filterDockerCLIOutput(&errSink),
 				TruncateOutput: printer.TruncateOutput,
 			}
 		}
@@ -113,7 +106,7 @@ func (f *ContainerFn) Run(reader io.Reader, writer io.Writer) error {
 	}
 
 	if errSink.Len() > 0 {
-		f.FnResult.Stderr = errSink.String()
+		f.FnResult.Stderr = filterDockerCLIOutput(&errSink)
 	}
 	return nil
 }
@@ -135,8 +128,15 @@ func (f *ContainerFn) getDockerCmd() (*exec.Cmd, context.CancelFunc) {
 		"--user", uidgid,
 		"--security-opt=no-new-privileges",
 	}
-	if f.ImagePullPolicy == NeverPull {
+	switch f.ImagePullPolicy {
+	case NeverPull:
 		args = append(args, "--pull", "never")
+	case AlwaysPull:
+		args = append(args, "--pull", "pull")
+	case IfNotPresentPull:
+		args = append(args, "--pull", "missing")
+	default:
+		args = append(args, "--pull", "missing")
 	}
 	for _, storageMount := range f.StorageMounts {
 		args = append(args, "--mount", storageMount.String())
@@ -173,62 +173,6 @@ func NewContainerEnvFromStringSlice(envStr []string) *runtimeutil.ContainerEnv {
 	return ce
 }
 
-// prepareImage will check local images and pull it if it doesn't
-// exist.
-func (f *ContainerFn) prepareImage() error {
-	// If ImagePullPolicy is set to "never", we don't need to do anything here.
-	if f.ImagePullPolicy == NeverPull {
-		return nil
-	}
-
-	// If ImagePullPolicy is set to "ifNotPresent", we scan the local images
-	// first. If there is a match, we just return. This can be useful for local
-	// development to prevent the remote image to accidentally override the
-	// local image when they use the same name and tag.
-	if f.ImagePullPolicy == IfNotPresentPull {
-		if foundInLocalCache := f.checkImageExistence(); foundInLocalCache {
-			return nil
-		}
-	}
-
-	// If ImagePullPolicy is set to always (which is the default), we will try
-	// to pull the image regardless if the tag has been seen in the local cache.
-	// This can help to ensure we have the latest release for "moving tags" like
-	// v1 and v1.2. The performance cost is very minimal, since `docker pull`
-	// checks the SHA first and only pull the missing docker layer(s).
-	args := []string{"image", "pull", f.Image}
-	// setup timeout
-	timeout := defaultLongTimeout
-	if f.Timeout != 0 {
-		timeout = f.Timeout
-	}
-	ctx, cancel := context.WithTimeout(context.Background(), timeout)
-	defer cancel()
-	cmd := exec.CommandContext(ctx, dockerBin, args...)
-	output, err := cmd.CombinedOutput()
-	if err != nil {
-		return &ContainerImageError{
-			Image:  f.Image,
-			Output: string(output),
-		}
-	}
-	return nil
-}
-
-// checkImageExistence returns true if the image does exist in
-// local cache
-func (f *ContainerFn) checkImageExistence() bool {
-	args := []string{"image", "inspect", f.Image}
-	ctx, cancel := context.WithTimeout(context.Background(), defaultShortTimeout)
-	defer cancel()
-	cmd := exec.CommandContext(ctx, dockerBin, args...)
-	if _, err := cmd.CombinedOutput(); err == nil {
-		// image exists locally
-		return true
-	}
-	return false
-}
-
 // AddDefaultImagePathPrefix adds default gcr.io/kpt-fn/ path prefix to image if only image name is specified
 func AddDefaultImagePathPrefix(image string) string {
 	if !strings.Contains(image, "/") {
@@ -250,3 +194,48 @@ func (e *ContainerImageError) Error() string {
 		"Error: Function image %q doesn't exist remotely. If you are developing new functions locally, you can choose to set the image pull policy to ifNotPresent or never.\n%v",
 		e.Image, e.Output)
 }
+
+// filterDockerCLIOutput filters out docker CLI messages
+// from the given buffer.
+func filterDockerCLIOutput(in io.Reader) string {
+	s := bufio.NewScanner(in)
+	var lines []string
+
+	for s.Scan() {
+		txt := s.Text()
+		if !isdockerCLIoutput(txt) {
+			lines = append(lines, txt)
+		}
+	}
+	return strings.Join(lines, "\n")
+}
+
+// isdockerCLIoutput is helper method to determine if
+// the given string is a docker CLI output message.
+// Example docker output:
+//	"Unable to find image 'gcr.io/kpt-fn/starlark:v0.3' locally"
+//  "v0.3: Pulling from kpt-fn/starlark"
+//  "4e9f2cdf4387: Already exists"
+//  "aafbf7df3ddf: Pulling fs layer"
+//  "aafbf7df3ddf: Verifying Checksum"
+//  "aafbf7df3ddf: Download complete"
+//  "6b759ab96cb2: Waiting"
+//  "aafbf7df3ddf: Pull complete"
+//  "Digest: sha256:c347e28606fa1a608e8e02e03541a5a46e4a0152005df4a11e44f6c4ab1edd9a"
+//  "Status: Downloaded newer image for gcr.io/kpt-fn/starlark:v0.3"
+//
+func isdockerCLIoutput(s string) bool {
+	if strings.Contains(s, ": Already exists") ||
+		strings.Contains(s, ": Pulling fs layer") ||
+		strings.Contains(s, ": Verifying Checksum") ||
+		strings.Contains(s, ": Download complete") ||
+		strings.Contains(s, ": Pulling from") ||
+		strings.Contains(s, ": Waiting") ||
+		strings.Contains(s, ": Pull complete") ||
+		strings.Contains(s, "Digest: sha256") ||
+		strings.Contains(s, "Status: Downloaded newer image") ||
+		strings.Contains(s, "Unable to find image") {
+		return true
+	}
+	return false
+}
diff --git a/site/reference/cli/fn/eval/README.md b/site/reference/cli/fn/eval/README.md
index d8c2e27d..287f5c0f 100644
--- a/site/reference/cli/fn/eval/README.md
+++ b/site/reference/cli/fn/eval/README.md
@@ -90,7 +90,7 @@ fn-args:
   Container image of the function to execute e.g. `gcr.io/kpt-fn/set-namespace:v0.1`.
   For convenience, if full image path is not specified, `gcr.io/kpt-fn/` is added as default prefix.
   e.g. instead of passing `gcr.io/kpt-fn/set-namespace:v0.1` you can pass `set-namespace:v0.1`.
-  `eval` executes only one function, so do not use `--exec-path` flag with this flag.
+  `eval` executes only one function, so do not use `--exec` flag with this flag.
 
 --image-pull-policy:
   If the image should be pulled before rendering the package(s). It can be set
diff --git a/thirdparty/cmdconfig/commands/cmdeval/cmdeval.go b/thirdparty/cmdconfig/commands/cmdeval/cmdeval.go
index e51bb2ab..73cee61b 100644
--- a/thirdparty/cmdconfig/commands/cmdeval/cmdeval.go
+++ b/thirdparty/cmdconfig/commands/cmdeval/cmdeval.go
@@ -61,7 +61,7 @@ func GetEvalFnRunner(ctx context.Context, parent string) *EvalFnRunner {
 		"a list of environment variables to be used by functions")
 	r.Command.Flags().BoolVar(
 		&r.AsCurrentUser, "as-current-user", false, "use the uid and gid that kpt is running with to run the function in the container")
-	r.Command.Flags().StringVar(&r.ImagePullPolicy, "image-pull-policy", string(fnruntime.AlwaysPull),
+	r.Command.Flags().StringVar(&r.ImagePullPolicy, "image-pull-policy", string(fnruntime.IfNotPresentPull),
 		fmt.Sprintf("pull image before running the container. It must be one of %s, %s and %s.", fnruntime.AlwaysPull, fnruntime.IfNotPresentPull, fnruntime.NeverPull))
 	r.Command.Flags().StringVar(
 		&r.Selector.APIVersion, "match-api-version", "", "select resources matching the given apiVersion")


Test patch:
diff --git a/e2e/live_test.go b/e2e/live_test.go
index 1ab91a6c..159c1e14 100644
--- a/e2e/live_test.go
+++ b/e2e/live_test.go
@@ -17,12 +17,12 @@
 package e2e
 
 import (
-	"fmt"
 	"os"
 	"path/filepath"
 	"testing"
 
 	livetest "github.com/GoogleContainerTools/kpt/pkg/test/live"
+	"github.com/stretchr/testify/require"
 )
 
 func TestLiveApply(t *testing.T) {
@@ -32,18 +32,40 @@ func TestLiveApply(t *testing.T) {
 func runTests(t *testing.T, path string) {
 	testCases := scanTestCases(t, path)
 
+	livetest.RemoveKindCluster(t)
+	livetest.CreateKindCluster(t)
+
 	for p := range testCases {
+		p := p
 		c := testCases[p]
-		for format := range c.Output {
-			testName := fmt.Sprintf("%s#%s", p, format)
-			t.Run(testName, func(t *testing.T) {
-				(&livetest.Runner{
-					Config: c,
-					Path: p,
-					Format: format,
-				}).Run(t)
-			})
+
+		if !c.Parallel {
+			continue
 		}
+
+		t.Run(p, func(t *testing.T) {
+			if c.Parallel {
+				t.Parallel()
+			}
+
+			if c.NoResourceGroup {
+				require.False(t, c.Parallel, "Parallel tests can not modify the RG CRD")
+				if livetest.CheckIfResourceGroupInstalled(t) {
+					livetest.RemoveResourceGroup(t)
+				}
+			} else {
+				livetest.InstallResourceGroup(t)
+			}
+
+			ns := filepath.Base(p)
+			livetest.CreateNamespace(t, ns)
+			defer livetest.RemoveNamespace(t, ns)
+
+			(&livetest.Runner{
+				Config:    c,
+				Path:      p,
+			}).Run(t)
+		})
 	}
 }
 
@@ -68,4 +90,4 @@ func scanTestCases(t *testing.T, path string) map[string]livetest.TestCaseConfig
 		t.Fatalf("failed to scan for test cases in %s", path)
 	}
 	return testCases
-}
\ No newline at end of file
+}
diff --git a/e2e/testdata/fn-eval/error-in-pipe/.expected/config.yaml b/e2e/testdata/fn-eval/error-in-pipe/.expected/config.yaml
index 21480468..77df92ae 100644
--- a/e2e/testdata/fn-eval/error-in-pipe/.expected/config.yaml
+++ b/e2e/testdata/fn-eval/error-in-pipe/.expected/config.yaml
@@ -21,8 +21,11 @@ stdErr: |
     Stderr:
       "[error] /// : failed to configure function: input namespace cannot be empty"
     Exit code: 1
-
+  
    
   [RUNNING] "gcr.io/kpt-fn/dne"
   [FAIL] "gcr.io/kpt-fn/dne" in 0s
-  Error: Function image "gcr.io/kpt-fn/dne" doesn't exist
\ No newline at end of file
+    Stderr:
+      "docker: Error response from daemon: manifest for gcr.io/kpt-fn/dne:latest not found: manifest unknown: Failed to fetch \"latest\" from request \"/v2/kpt-fn/dne/manifests/latest\"."
+      "See 'docker run --help'."
+    Exit code: 125
\ No newline at end of file
diff --git a/e2e/testdata/fn-eval/fn-success-with-stderr/pkg/.expected/results.yaml b/e2e/testdata/fn-eval/fn-success-with-stderr/pkg/.expected/results.yaml
index cd73b3c4..67f0c37a 100644
--- a/e2e/testdata/fn-eval/fn-success-with-stderr/pkg/.expected/results.yaml
+++ b/e2e/testdata/fn-eval/fn-success-with-stderr/pkg/.expected/results.yaml
@@ -5,6 +5,5 @@ metadata:
 exitCode: 0
 items:
   - image: gcr.io/kpt-fn/starlark:v0.2
-    stderr: |
-      function succeeded, reporting it on stderr
+    stderr: function succeeded, reporting it on stderr
     exitCode: 0
diff --git a/e2e/testdata/fn-eval/missing-fn-image/.expected/config.yaml b/e2e/testdata/fn-eval/missing-fn-image/.expected/config.yaml
index cbef6dd6..fe98cda9 100644
--- a/e2e/testdata/fn-eval/missing-fn-image/.expected/config.yaml
+++ b/e2e/testdata/fn-eval/missing-fn-image/.expected/config.yaml
@@ -17,4 +17,4 @@ exitCode: 1
 image: gcr.io/kpt-fn/dne # non-existing image
 args:
   namespace: staging
-stdErr: 'Function image "gcr.io/kpt-fn/dne" doesn''t exist'
+stdErr: 'gcr.io/kpt-fn/dne:latest not found'
diff --git a/e2e/testdata/fn-render/fn-success-with-stderr/.expected/results.yaml b/e2e/testdata/fn-render/fn-success-with-stderr/.expected/results.yaml
index cd73b3c4..67f0c37a 100644
--- a/e2e/testdata/fn-render/fn-success-with-stderr/.expected/results.yaml
+++ b/e2e/testdata/fn-render/fn-success-with-stderr/.expected/results.yaml
@@ -5,6 +5,5 @@ metadata:
 exitCode: 0
 items:
   - image: gcr.io/kpt-fn/starlark:v0.2
-    stderr: |
-      function succeeded, reporting it on stderr
+    stderr: function succeeded, reporting it on stderr
     exitCode: 0
diff --git a/e2e/testdata/fn-render/missing-fn-image/.expected/config.yaml b/e2e/testdata/fn-render/missing-fn-image/.expected/config.yaml
index fab1748a..7bee7e35 100644
--- a/e2e/testdata/fn-render/missing-fn-image/.expected/config.yaml
+++ b/e2e/testdata/fn-render/missing-fn-image/.expected/config.yaml
@@ -13,4 +13,4 @@
 # limitations under the License.
 
 exitCode: 1
-stdErr: 'Error: Function image "gcr.io/kpt-fn/dne" doesn''t exist'
+stdErr: 'gcr.io/kpt-fn/dne:latest not found'
diff --git a/e2e/testdata/live-apply/apply-depends-on/config.yaml b/e2e/testdata/live-apply/apply-depends-on/config.yaml
index 51421563..c12c7ab5 100644
--- a/e2e/testdata/live-apply/apply-depends-on/config.yaml
+++ b/e2e/testdata/live-apply/apply-depends-on/config.yaml
@@ -12,20 +12,17 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-preinstallResourceGroup: true
-output:
-  events:
-    stdOut: |
-      deployment.apps/first-nginx created
-      1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
-      deployment.apps/second-nginx created
-      1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
-  json:
-    stdOut: |
-      {"eventType":"resourceApplied","group":"apps","kind":"Deployment","name":"first-nginx","namespace":"apply-depends-on","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
-      {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
-      {"eventType":"resourceApplied","group":"apps","kind":"Deployment","name":"second-nginx","namespace":"apply-depends-on","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
-      {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
+parallel: true
+
+kptArgs:
+  - "--reconcile-timeout=1m"
+
+stdOut: |
+  deployment.apps/first-nginx created
+  1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
+  deployment.apps/second-nginx created
+  1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
+
 inventory:
   - group: apps
     kind: Deployment
diff --git a/e2e/testdata/live-apply/apply-depends-on/resources/first.yaml b/e2e/testdata/live-apply/apply-depends-on/resources/first.yaml
index 00bbf522..b5a95d23 100644
--- a/e2e/testdata/live-apply/apply-depends-on/resources/first.yaml
+++ b/e2e/testdata/live-apply/apply-depends-on/resources/first.yaml
@@ -18,7 +18,7 @@ metadata:
   name: first-nginx
   namespace: apply-depends-on
 spec:
-  replicas: 2
+  replicas: 1
   selector:
     matchLabels:
       app: first-nginx
diff --git a/e2e/testdata/live-apply/apply-depends-on/resources/second.yaml b/e2e/testdata/live-apply/apply-depends-on/resources/second.yaml
index 64e5dcc6..f3dfb252 100644
--- a/e2e/testdata/live-apply/apply-depends-on/resources/second.yaml
+++ b/e2e/testdata/live-apply/apply-depends-on/resources/second.yaml
@@ -20,7 +20,7 @@ metadata:
   annotations:
     config.kubernetes.io/depends-on: apps/namespaces/apply-depends-on/Deployment/first-nginx
 spec:
-  replicas: 2
+  replicas: 1
   selector:
     matchLabels:
       app: second-nginx
diff --git a/e2e/testdata/live-apply/crd-and-cr/config.yaml b/e2e/testdata/live-apply/crd-and-cr/config.yaml
index 4db0e064..943b973a 100644
--- a/e2e/testdata/live-apply/crd-and-cr/config.yaml
+++ b/e2e/testdata/live-apply/crd-and-cr/config.yaml
@@ -12,21 +12,16 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-requiresCleanCluster: true
-preinstallResourceGroup: true
-output:
-  events:
-    stdOut: |
-      customresourcedefinition.apiextensions.k8s.io/customs.kpt.dev created
-      1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
-      custom.kpt.dev/cr created
-      1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
-  json:
-    stdOut: |
-      {"eventType":"resourceApplied","group":"apiextensions.k8s.io","kind":"CustomResourceDefinition","name":"customs.kpt.dev","namespace":"","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
-      {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
-      {"eventType":"resourceApplied","group":"kpt.dev","kind":"Custom","name":"cr","namespace":"","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
-      {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
+parallel: true
+
+kptArgs:
+  - "--reconcile-timeout=1m"
+
+stdOut: |
+  customresourcedefinition.apiextensions.k8s.io/customs.kpt.dev created
+  1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
+  custom.kpt.dev/cr created
+  1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
 inventory:
   - group: apiextensions.k8s.io
     kind: CustomResourceDefinition
diff --git a/e2e/testdata/live-apply/dry-run-with-install-rg/config.yaml b/e2e/testdata/live-apply/dry-run-with-install-rg/config.yaml
index 7f10c3e8..5f8ab9af 100644
--- a/e2e/testdata/live-apply/dry-run-with-install-rg/config.yaml
+++ b/e2e/testdata/live-apply/dry-run-with-install-rg/config.yaml
@@ -12,21 +12,14 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-requiresCleanCluster: true
+parallel: false
+noResourceGroup: true
 kptArgs:
   - "--dry-run"
   - "--install-resource-group"
-output:
-  events:
-    stdOut: |
-      deployment.apps/nginx-deployment created (dry-run)
-      1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed (dry-run)
-    stdErr: |
-      installing inventory ResourceGroup CRD.
-  json:
-    stdOut: |
-      {"eventType":"resourceApplied","group":"apps","kind":"Deployment","name":"nginx-deployment","namespace":"dry-run-with-install-rg","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
-      {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
-    stdErr: |
-      installing inventory ResourceGroup CRD.
+stdOut: |
+  deployment.apps/nginx-deployment created (dry-run)
+  1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed (dry-run)
+stdErr: |
+  installing inventory ResourceGroup CRD.
 exitCode: 0
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/dry-run-with-install-rg/resources/deployment.yaml b/e2e/testdata/live-apply/dry-run-with-install-rg/resources/deployment.yaml
index d37e0895..38c79d76 100644
--- a/e2e/testdata/live-apply/dry-run-with-install-rg/resources/deployment.yaml
+++ b/e2e/testdata/live-apply/dry-run-with-install-rg/resources/deployment.yaml
@@ -24,7 +24,7 @@ spec:
     rollingUpdate:
       maxUnavailable: 1
       maxSurge: 0
-  replicas: 2
+  replicas: 1
   selector:
     matchLabels:
       app: nginx-deployment
diff --git a/e2e/testdata/live-apply/dry-run-without-rg/config.yaml b/e2e/testdata/live-apply/dry-run-without-rg/config.yaml
index b89bf9c9..608a7e83 100644
--- a/e2e/testdata/live-apply/dry-run-without-rg/config.yaml
+++ b/e2e/testdata/live-apply/dry-run-without-rg/config.yaml
@@ -12,11 +12,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-requiresCleanCluster: true
+parallel: false
+noResourceGroup: true
 kptArgs:
   - "--dry-run"
-output:
-  events:
-    stdErr: |
-      Error: The ResourceGroup CRD was not found in the cluster. Please install it either by using the '--install-resource-group' flag or the 'kpt live install-resource-group' command.
+stdErr: |
+  Error: The ResourceGroup CRD was not found in the cluster. Please install it either by using the '--install-resource-group' flag or the 'kpt live install-resource-group' command.
 exitCode: 1
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/install-rg-on-apply/config.yaml b/e2e/testdata/live-apply/install-rg-on-apply/config.yaml
index 207b4a45..b0019b96 100644
--- a/e2e/testdata/live-apply/install-rg-on-apply/config.yaml
+++ b/e2e/testdata/live-apply/install-rg-on-apply/config.yaml
@@ -12,20 +12,15 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-requiresCleanCluster: true
-output:
-  events:
-    stdOut: |
-      deployment.apps/nginx-deployment created
-      1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
-    stdErr: |
-      installing inventory ResourceGroup CRD.
-  json:
-    stdOut: |
-      {"eventType":"resourceApplied","group":"apps","kind":"Deployment","name":"nginx-deployment","namespace":"install-rg-on-apply","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
-      {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
-    stdErr: |
-      installing inventory ResourceGroup CRD.
+parallel: false
+noResourceGroup: true
+kptArgs:
+  - "--reconcile-timeout=1m"
+stdOut: |
+  deployment.apps/nginx-deployment created
+  1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
+stdErr: |
+  installing inventory ResourceGroup CRD.
 inventory:
   - group: apps
     kind: Deployment
diff --git a/e2e/testdata/live-apply/install-rg-on-apply/resources/deployment.yaml b/e2e/testdata/live-apply/install-rg-on-apply/resources/deployment.yaml
index 247e4fd6..1b521f24 100644
--- a/e2e/testdata/live-apply/install-rg-on-apply/resources/deployment.yaml
+++ b/e2e/testdata/live-apply/install-rg-on-apply/resources/deployment.yaml
@@ -24,7 +24,7 @@ spec:
     rollingUpdate:
       maxUnavailable: 1
       maxSurge: 0
-  replicas: 2
+  replicas: 1
   selector:
     matchLabels:
       app: nginx-deployment
diff --git a/e2e/testdata/live-apply/json-output/config.yaml b/e2e/testdata/live-apply/json-output/config.yaml
new file mode 100644
index 00000000..1d950b5f
--- /dev/null
+++ b/e2e/testdata/live-apply/json-output/config.yaml
@@ -0,0 +1,35 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+parallel: true
+kptArgs:
+  - "--output=json"
+  - "--reconcile-timeout=1m"
+stdOut: |
+  {"eventType":"resourceApplied","group":"","kind":"ConfigMap","name":"cm","namespace":"json-output","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
+  {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
+  {"eventType":"resourceApplied","group":"apps","kind":"Deployment","name":"nginx","namespace":"json-output","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
+  {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
+  {"eventType":"resourcePruned","group":"apps","kind":"Deployment","name":"second-nginx","namespace":"json-output","operation":"Pruned","timestamp":"<TIMESTAMP>","type":"prune"}
+  {"eventType":"completed","pruned":1,"skipped":0,"timestamp":"<TIMESTAMP>","type":"prune"}
+  {"eventType":"resourcePruned","group":"apps","kind":"Deployment","name":"first-nginx","namespace":"json-output","operation":"Pruned","timestamp":"<TIMESTAMP>","type":"prune"}
+  {"eventType":"completed","pruned":1,"skipped":0,"timestamp":"<TIMESTAMP>","type":"prune"}
+inventory:
+  - kind: ConfigMap
+    name: cm
+    namespace: json-output
+  - group: apps
+    kind: Deployment
+    name: nginx
+    namespace: json-output
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/json-output/pre-apply/first.yaml b/e2e/testdata/live-apply/json-output/pre-apply/first.yaml
new file mode 100644
index 00000000..f671e0b9
--- /dev/null
+++ b/e2e/testdata/live-apply/json-output/pre-apply/first.yaml
@@ -0,0 +1,36 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: first-nginx
+  namespace: json-output
+  annotations:
+    config.k8s.io/owning-inventory: json-output
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: first-nginx
+  template:
+    metadata:
+      labels:
+        app: first-nginx
+    spec:
+      containers:
+        - name: nginx
+          image: nginx:1.14.2
+          ports:
+            - containerPort: 80
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/json-output/pre-apply/rg.yaml b/e2e/testdata/live-apply/json-output/pre-apply/rg.yaml
new file mode 100644
index 00000000..506bc71b
--- /dev/null
+++ b/e2e/testdata/live-apply/json-output/pre-apply/rg.yaml
@@ -0,0 +1,31 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: kpt.dev/v1alpha1
+kind: ResourceGroup
+metadata:
+  labels:
+    cli-utils.sigs.k8s.io/inventory-id: json-output
+  name: json-output
+  namespace: json-output
+spec:
+  resources:
+    - group: apps
+      kind: Deployment
+      name: first-nginx
+      namespace: json-output
+    - group: apps
+      kind: Deployment
+      name: second-nginx
+      namespace: json-output
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/json-output/pre-apply/second.yaml b/e2e/testdata/live-apply/json-output/pre-apply/second.yaml
new file mode 100644
index 00000000..3c4fdfa8
--- /dev/null
+++ b/e2e/testdata/live-apply/json-output/pre-apply/second.yaml
@@ -0,0 +1,37 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: second-nginx
+  namespace: json-output
+  annotations:
+    config.k8s.io/owning-inventory: json-output
+    config.kubernetes.io/depends-on: apps/namespaces/json-output/Deployment/first-nginx
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: second-nginx
+  template:
+    metadata:
+      labels:
+        app: second-nginx
+    spec:
+      containers:
+        - name: nginx
+          image: nginx:1.14.2
+          ports:
+            - containerPort: 80
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/json-output/resources/Kptfile b/e2e/testdata/live-apply/json-output/resources/Kptfile
new file mode 100644
index 00000000..dff6968e
--- /dev/null
+++ b/e2e/testdata/live-apply/json-output/resources/Kptfile
@@ -0,0 +1,8 @@
+apiVersion: kpt.dev/v1
+kind: Kptfile
+metadata:
+  name: json-output
+inventory:
+  namespace: json-output
+  name: json-output
+  inventoryID: json-output
diff --git a/e2e/testdata/live-apply/json-output/resources/cm.yaml b/e2e/testdata/live-apply/json-output/resources/cm.yaml
new file mode 100644
index 00000000..153818fe
--- /dev/null
+++ b/e2e/testdata/live-apply/json-output/resources/cm.yaml
@@ -0,0 +1,21 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cm
+  namespace: json-output
+data:
+  foo: bar
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/json-output/resources/dep.yaml b/e2e/testdata/live-apply/json-output/resources/dep.yaml
new file mode 100644
index 00000000..eb0129b4
--- /dev/null
+++ b/e2e/testdata/live-apply/json-output/resources/dep.yaml
@@ -0,0 +1,36 @@
+# Copyright 2021 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: nginx
+  namespace: json-output
+  annotations:
+    config.kubernetes.io/depends-on: apps/namespaces/json-output/ConfigMap/cm
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: nginx
+  template:
+    metadata:
+      labels:
+        app: nginx
+    spec:
+      containers:
+        - name: nginx
+          image: nginx:1.14.2
+          ports:
+            - containerPort: 80
\ No newline at end of file
diff --git a/e2e/testdata/live-apply/prune-depends-on/config.yaml b/e2e/testdata/live-apply/prune-depends-on/config.yaml
index 6a96ac6a..91e57f40 100644
--- a/e2e/testdata/live-apply/prune-depends-on/config.yaml
+++ b/e2e/testdata/live-apply/prune-depends-on/config.yaml
@@ -12,24 +12,18 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-preinstallResourceGroup: true
-output:
-  events:
-    stdOut: |
-      configmap/cm created
-      1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
-      deployment.apps/second-nginx pruned
-      1 resource(s) pruned, 0 skipped, 0 failed
-      deployment.apps/first-nginx pruned
-      1 resource(s) pruned, 0 skipped, 0 failed
-  json:
-    stdOut: |
-      {"eventType":"resourceApplied","group":"","kind":"ConfigMap","name":"cm","namespace":"prune-depends-on","operation":"Created","timestamp":"<TIMESTAMP>","type":"apply"}
-      {"configuredCount":0,"count":1,"createdCount":1,"eventType":"completed","failedCount":0,"serverSideCount":0,"timestamp":"<TIMESTAMP>","type":"apply","unchangedCount":0}
-      {"eventType":"resourcePruned","group":"apps","kind":"Deployment","name":"second-nginx","namespace":"prune-depends-on","operation":"Pruned","timestamp":"<TIMESTAMP>","type":"prune"}
-      {"eventType":"completed","pruned":1,"skipped":0,"timestamp":"<TIMESTAMP>","type":"prune"}
-      {"eventType":"resourcePruned","group":"apps","kind":"Deployment","name":"first-nginx","namespace":"prune-depends-on","operation":"Pruned","timestamp":"<TIMESTAMP>","type":"prune"}
-      {"eventType":"completed","pruned":1,"skipped":0,"timestamp":"<TIMESTAMP>","type":"prune"}
+parallel: true
+
+kptArgs:
+  - "--reconcile-timeout=1m"
+
+stdOut: |
+  configmap/cm created
+  1 resource(s) applied. 1 created, 0 unchanged, 0 configured, 0 failed
+  deployment.apps/second-nginx pruned
+  1 resource(s) pruned, 0 skipped, 0 failed
+  deployment.apps/first-nginx pruned
+  1 resource(s) pruned, 0 skipped, 0 failed
 inventory:
   - kind: ConfigMap
     name: cm
diff --git a/e2e/testdata/live-apply/prune-depends-on/pre-apply/first.yaml b/e2e/testdata/live-apply/prune-depends-on/pre-apply/first.yaml
index 16c645d4..5097c0af 100644
--- a/e2e/testdata/live-apply/prune-depends-on/pre-apply/first.yaml
+++ b/e2e/testdata/live-apply/prune-depends-on/pre-apply/first.yaml
@@ -20,7 +20,7 @@ metadata:
   annotations:
     config.k8s.io/owning-inventory: prune-depends-on
 spec:
-  replicas: 2
+  replicas: 1
   selector:
     matchLabels:
       app: first-nginx
diff --git a/e2e/testdata/live-apply/prune-depends-on/pre-apply/second.yaml b/e2e/testdata/live-apply/prune-depends-on/pre-apply/second.yaml
index f592733d..238a51aa 100644
--- a/e2e/testdata/live-apply/prune-depends-on/pre-apply/second.yaml
+++ b/e2e/testdata/live-apply/prune-depends-on/pre-apply/second.yaml
@@ -21,7 +21,7 @@ metadata:
     config.k8s.io/owning-inventory: prune-depends-on
     config.kubernetes.io/depends-on: apps/namespaces/prune-depends-on/Deployment/first-nginx
 spec:
-  replicas: 2
+  replicas: 1
   selector:
     matchLabels:
       app: second-nginx
diff --git a/internal/fnruntime/container_test.go b/internal/fnruntime/container_test.go
index 11161f1c..f6ac50ac 100644
--- a/internal/fnruntime/container_test.go
+++ b/internal/fnruntime/container_test.go
@@ -24,6 +24,7 @@ import (
 
 	"github.com/GoogleContainerTools/kpt/internal/fnruntime"
 	"github.com/GoogleContainerTools/kpt/internal/printer"
+	fnresult "github.com/GoogleContainerTools/kpt/pkg/api/fnresult/v1"
 	"github.com/stretchr/testify/assert"
 )
 
@@ -54,6 +55,9 @@ func TestContainerFn(t *testing.T) {
 			instance := fnruntime.ContainerFn{
 				Ctx:   printer.WithContext(ctx, printer.New(nil, errBuff)),
 				Image: tt.image,
+				FnResult: &fnresult.Result{
+					Image: tt.image,
+				},
 			}
 			input := bytes.NewBufferString(tt.input)
 			output := &bytes.Buffer{}
diff --git a/internal/fnruntime/fnerrors_test.go b/internal/fnruntime/fnerrors_test.go
index b022723b..b308d721 100644
--- a/internal/fnruntime/fnerrors_test.go
+++ b/internal/fnruntime/fnerrors_test.go
@@ -15,6 +15,7 @@
 package fnruntime
 
 import (
+	"bytes"
 	"testing"
 
 	"github.com/stretchr/testify/assert"
@@ -138,3 +139,76 @@ error message`,
 		})
 	}
 }
+
+func TestDockerCLIOutputFilter(t *testing.T) {
+
+	testcases := []struct {
+		name     string
+		input    string
+		expected string
+	}{
+		{
+			name: "should filter docker CLI output successfully",
+			input: `Unable to find image 'gcr.io/kpt-fn/starlark:v0.3' locally
+v0.3: Pulling from kpt-fn/starlark
+4e9f2cdf4387: Already exists
+aafbf7df3ddf: Pulling fs layer
+aafbf7df3ddf: Verifying Checksum
+aafbf7df3ddf: Download complete
+aafbf7df3ddf: Pull complete
+6b759ab96cb2: Waiting
+Digest: sha256:c347e28606fa1a608e8e02e03541a5a46e4a0152005df4a11e44f6c4ab1edd9a
+Status: Downloaded newer image for gcr.io/kpt-fn/starlark:v0.3
+`,
+			expected: "",
+		},
+		{
+			name: "should filter docker messages and shouldn't truncate trailing lines",
+			input: `Unable to find image 'gcr.io/kpt-fn/starlark:v0.3' locally
+v0.3: Pulling from kpt-fn/starlark
+4e9f2cdf4387: Already exists
+aafbf7df3ddf: Pulling fs layer
+aafbf7df3ddf: Verifying Checksum
+aafbf7df3ddf: Download complete
+aafbf7df3ddf: Pull complete
+6b759ab96cb2: Waiting
+Digest: sha256:c347e28606fa1a608e8e02e03541a5a46e4a0152005df4a11e44f6c4ab1edd9a
+Status: Downloaded newer image for gcr.io/kpt-fn/starlark:v0.3
+line before last line
+lastline
+
+`,
+			expected: `line before last line
+lastline
+`,
+		},
+		{
+			name: "should filter interleaved docker messages",
+			input: `firstline
+Unable to find image 'gcr.io/kpt-fn/starlark:v0.3' locally
+v0.3: Pulling from kpt-fn/starlark
+4e9f2cdf4387: Already exists
+aafbf7df3ddf: Pulling fs layer
+aafbf7df3ddf: Verifying Checksum
+line in the middle
+aafbf7df3ddf: Download complete
+aafbf7df3ddf: Pull complete
+6b759ab96cb2: Waiting
+Digest: sha256:c347e28606fa1a608e8e02e03541a5a46e4a0152005df4a11e44f6c4ab1edd9a
+Status: Downloaded newer image for gcr.io/kpt-fn/starlark:v0.3
+lastline
+`,
+			expected: `firstline
+line in the middle
+lastline`,
+		},
+	}
+
+	for _, tc := range testcases {
+		tc := tc
+		t.Run(tc.name, func(t *testing.T) {
+			s := filterDockerCLIOutput(bytes.NewBufferString(tc.input))
+			assert.Equal(t, tc.expected, s)
+		})
+	}
+}
diff --git a/pkg/test/live/config.go b/pkg/test/live/config.go
index 174f15a1..d55cf750 100644
--- a/pkg/test/live/config.go
+++ b/pkg/test/live/config.go
@@ -26,33 +26,30 @@ type TestCaseConfig struct {
 	// ExitCode is the expected exit code from the kpt commands. Default: 0
 	ExitCode int `yaml:"exitCode,omitempty"`
 
-	Output map[string]Output `yaml:"output,omitempty"`
+	// StdErr is the expected standard error output. Default: ""
+	StdErr string `yaml:"stdErr,omitempty"`
+
+	// StdOut is the expected standard output from running the command.
+	// Default: ""
+	StdOut string `yaml:"stdOut,omitempty"`
 
 	// Inventory is the expected list of resource present in the inventory.
 	Inventory []InventoryEntry `yaml:"inventory,omitempty"`
 
-	// RequiresCleanCluster tells the test framework that a new cluster must
-	// be created for running this test.
-	RequiresCleanCluster bool `yaml:"requiresCleanCluster,omitempty"`
+	// NoResourceGroup defines whether the RG CRD should be present in the cluster
+	// when the test starts.
+	NoResourceGroup bool `yaml:"noResourceGroup,omitempty"`
 
-	// PreinstallResourceGroup causes the test framework to verify that the
-	// ResourceGroup CRD is available in the cluster before running the test.
-	PreinstallResourceGroup bool `yaml:"preinstallResourceGroup,omitempty"`
+	// Parallel defines whether the test can be run in parallel with other
+	// tests. The primary requirement here is that the test doesn't create,
+	// update, or delete any cluster-scoped resources.
+	Parallel bool `yaml:"parallel,omitempty"`
 
 	// KptArgs is a list of args that will be provided to the kpt command
 	// when running the test.
 	KptArgs []string `yaml:"kptArgs,omitempty"`
 }
 
-type Output struct {
-	// StdErr is the expected standard error output. Default: ""
-	StdErr string `yaml:"stdErr,omitempty"`
-
-	// StdOut is the expected standard output from running the command.
-	// Default: ""
-	StdOut string `yaml:"stdOut,omitempty"`
-}
-
 // InventoryEntry defines an entry in an inventory list.
 type InventoryEntry struct {
 	Group     string `yaml:"group,omitempty"`
diff --git a/pkg/test/live/kind.go b/pkg/test/live/kind.go
new file mode 100644
index 00000000..10780562
--- /dev/null
+++ b/pkg/test/live/kind.go
@@ -0,0 +1,111 @@
+// Copyright 2021 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package live
+
+import (
+	"bufio"
+	"bytes"
+	"fmt"
+	"os"
+	"os/exec"
+	"strings"
+	"testing"
+)
+
+const (
+	KindClusterName   = "live-e2e-test"
+	K8sVersionEnvName = "K8S_VERSION"
+)
+
+func InstallResourceGroup(t *testing.T) {
+	cmd := exec.Command("kpt", "live", "install-resource-group")
+	if err := cmd.Run(); err != nil {
+		t.Fatalf("error installing ResourceGroup CRD: %v", err)
+	}
+}
+
+func RemoveResourceGroup(t *testing.T) {
+	cmd := exec.Command("kubectl", "delete", "crd", "resourcegroups.kpt.dev")
+	if err := cmd.Run(); err != nil {
+		t.Fatalf("error removing ResourceGroup CRD: %v", err)
+	}
+	if CheckIfResourceGroupInstalled(t) {
+		t.Fatalf("couldn't remove ResourceGroup CRD")
+	}
+}
+
+func CheckIfResourceGroupInstalled(t *testing.T) bool {
+	cmd := exec.Command("kubectl", "get", "resourcegroups.kpt.dev")
+	output, err := cmd.CombinedOutput()
+	if strings.Contains(string(output), "the server doesn't have a resource type") {
+		return false
+	}
+	if err != nil {
+		t.Fatalf("error checking for ResourceGroup CRD: %v", err)
+	}
+	return true
+}
+
+func CheckKindClusterAvailable(t *testing.T) bool {
+	cmd := exec.Command("kind", "get", "clusters")
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		t.Fatalf("failed to check for kind cluster: %v", err)
+	}
+
+	sc := bufio.NewScanner(bytes.NewReader(output))
+	for sc.Scan() {
+		if strings.TrimSpace(sc.Text()) == KindClusterName {
+			return true
+		}
+	}
+	if err := sc.Err(); err != nil {
+		t.Fatalf("error parsing output from 'kind get cluster': %v", err)
+	}
+	return false
+}
+
+func CreateKindCluster(t *testing.T) {
+	args := []string{"create", "cluster", fmt.Sprintf("--name=%s", KindClusterName)}
+	if k8sVersion := os.Getenv(K8sVersionEnvName); k8sVersion != "" {
+		t.Logf("Using version %s", k8sVersion)
+		args = append(args, fmt.Sprintf("--image=kindest/node:v%s", k8sVersion))
+	}
+	cmd := exec.Command("kind", args...)
+	if err := cmd.Run(); err != nil {
+		t.Fatalf("failed to create new kind cluster: %v", err)
+	}
+}
+
+func RemoveKindCluster(t *testing.T) {
+	cmd := exec.Command("kind", "delete", "cluster", fmt.Sprintf("--name=%s", KindClusterName))
+	if err := cmd.Run(); err != nil {
+		t.Fatalf("failed to remove existing cluster: %v", err)
+	}
+}
+
+func CreateNamespace(t *testing.T, namespace string) {
+	cmd := exec.Command("kubectl", "create", "ns", namespace)
+	if err := cmd.Run(); err != nil {
+		t.Fatalf("error creating namespace %s: %v", namespace, err)
+	}
+}
+
+func RemoveNamespace(t *testing.T, namespace string) {
+	cmd := exec.Command("kubectl", "delete", "ns", namespace, "--wait=false")
+	if err := cmd.Run(); err != nil {
+		t.Logf("error deleting namespace %s: %v", namespace, err)
+	}
+}
diff --git a/pkg/test/live/runner.go b/pkg/test/live/runner.go
index d4720f0b..304644ab 100644
--- a/pkg/test/live/runner.go
+++ b/pkg/test/live/runner.go
@@ -18,7 +18,6 @@ import (
 	"bufio"
 	"bytes"
 	"errors"
-	"fmt"
 	"os"
 	"os/exec"
 	"path/filepath"
@@ -28,14 +27,10 @@ import (
 	"testing"
 
 	"github.com/stretchr/testify/assert"
+	"sigs.k8s.io/cli-utils/pkg/kstatus/status"
 	"sigs.k8s.io/kustomize/kyaml/yaml"
 )
 
-const (
-	KindClusterName   = "live-e2e-test"
-	K8sVersionEnvName = "K8S_VERSION"
-)
-
 // Runner uses the provided Config to run a test.
 type Runner struct {
 	// Config provides the configuration for how this test should be
@@ -44,41 +39,11 @@ type Runner struct {
 
 	// Path provides the path to the test files.
 	Path string
-
-	// Format provides the output format that should be used.
-	Format string
 }
 
 // Run executes the test.
 func (r *Runner) Run(t *testing.T) {
 	testName := filepath.Base(r.Path)
-	isAvailable := r.CheckKindClusterAvailable(t)
-	if r.Config.RequiresCleanCluster {
-		t.Log("Test requires clean cluster")
-		if isAvailable {
-			t.Log("Removing existing cluster")
-			r.RemoveKindCluster(t)
-		}
-		t.Log("Creating new cluster")
-		r.CreateKindCluster(t)
-	} else {
-		if !isAvailable {
-			t.Log("Creating new cluster")
-			r.CreateKindCluster(t)
-		} else if r.CheckForNamespace(t, testName) {
-			t.Log("Namespace already exist, creating new cluster")
-			r.RemoveKindCluster(t)
-			r.CreateKindCluster(t)
-		}
-	}
-
-	if r.Config.PreinstallResourceGroup {
-		r.InstallResourceGroup(t)
-	}
-
-	r.CreateNamespace(t, testName)
-	defer r.RemoveNamespace(t, testName)
-
 	r.RunPreApply(t)
 
 	stdout, stderr, err := r.RunApply(t)
@@ -108,9 +73,6 @@ func (r *Runner) RunPreApply(t *testing.T) {
 
 func (r *Runner) RunApply(t *testing.T) (string, string, error) {
 	args := append([]string{"live", "apply"}, r.Config.KptArgs...)
-	if r.Format != "" {
-		args = append(args, "--output", r.Format)
-	}
 	t.Logf("Running command: kpt %s", strings.Join(args, " "))
 	cmd := exec.Command("kpt", args...)
 	cmd.Dir = filepath.Join(r.Path, "resources")
@@ -124,98 +86,6 @@ func (r *Runner) RunApply(t *testing.T) (string, string, error) {
 	return outBuf.String(), errBuf.String(), err
 }
 
-func (r *Runner) InstallResourceGroup(t *testing.T) {
-	cmd := exec.Command("kpt", "live", "install-resource-group")
-	if err := cmd.Run(); err != nil {
-		t.Fatalf("error installing ResourceGroup CRD: %v", err)
-	}
-}
-
-func (r *Runner) CheckIfResourceGroupInstalled(t *testing.T) bool {
-	cmd := exec.Command("kubectl", "get", "resourcegroups.kpt.dev")
-	output, err := cmd.CombinedOutput()
-	if strings.Contains(string(output), "the server doesn't have a resource type") {
-		return false
-	}
-	if err != nil {
-		t.Fatalf("error checking for ResourceGroup CRD: %v", err)
-	}
-	return true
-}
-
-func (r *Runner) CheckKindClusterAvailable(t *testing.T) bool {
-	cmd := exec.Command("kind", "get", "clusters")
-	output, err := cmd.CombinedOutput()
-	if err != nil {
-		t.Fatalf("failed to check for kind cluster: %v", err)
-	}
-
-	sc := bufio.NewScanner(bytes.NewReader(output))
-	for sc.Scan() {
-		if strings.TrimSpace(sc.Text()) == KindClusterName {
-			return true
-		}
-	}
-	if err := sc.Err(); err != nil {
-		t.Fatalf("error parsing output from 'kind get cluster': %v", err)
-	}
-	return false
-}
-
-func (r *Runner) CreateKindCluster(t *testing.T) {
-	args := []string{"create", "cluster", fmt.Sprintf("--name=%s", KindClusterName)}
-	if k8sVersion := os.Getenv(K8sVersionEnvName); k8sVersion != "" {
-		t.Logf("Using version %s", k8sVersion)
-		args = append(args, fmt.Sprintf("--image=kindest/node:v%s", k8sVersion))
-	}
-	cmd := exec.Command("kind", args...)
-	if err := cmd.Run(); err != nil {
-		t.Fatalf("failed to create new kind cluster: %v", err)
-	}
-}
-
-func (r *Runner) RemoveKindCluster(t *testing.T) {
-	cmd := exec.Command("kind", "delete", "cluster", fmt.Sprintf("--name=%s", KindClusterName))
-	if err := cmd.Run(); err != nil {
-		t.Fatalf("failed to remove existing cluster: %v", err)
-	}
-}
-
-func (r *Runner) CheckForNamespace(t *testing.T, namespace string) bool {
-	cmd := exec.Command("kubectl", "get", "ns", namespace, "--no-headers", "--output=name")
-	output, err := cmd.CombinedOutput()
-	if strings.Contains(string(output), "NotFound") {
-		return false
-	}
-	if err != nil {
-		t.Fatalf("error listing namespaces with kubectl: %v", err)
-	}
-	sc := bufio.NewScanner(bytes.NewReader(output))
-	for sc.Scan() {
-		if strings.TrimSpace(sc.Text()) == fmt.Sprintf("namespace/%s", namespace) {
-			return true
-		}
-	}
-	if err := sc.Err(); err != nil {
-		t.Fatalf("error parsing output from 'kubectl get ns': %v", err)
-	}
-	return false
-}
-
-func (r *Runner) CreateNamespace(t *testing.T, namespace string) {
-	cmd := exec.Command("kubectl", "create", "ns", namespace)
-	if err := cmd.Run(); err != nil {
-		t.Fatalf("error creating namespace %s: %v", namespace, err)
-	}
-}
-
-func (r *Runner) RemoveNamespace(t *testing.T, namespace string) {
-	cmd := exec.Command("kubectl", "delete", "ns", namespace, "--wait=false")
-	if err := cmd.Run(); err != nil {
-		t.Logf("error deleting namespace %s: %v", namespace, err)
-	}
-}
-
 func (r *Runner) VerifyExitCode(t *testing.T, err error) {
 	exitCode := 0
 	var exitErr *exec.ExitError
@@ -228,11 +98,15 @@ func (r *Runner) VerifyExitCode(t *testing.T, err error) {
 }
 
 func (r *Runner) VerifyStdout(t *testing.T, stdout string) {
-	assert.Equal(t, strings.TrimSpace(r.Config.Output[r.Format].StdOut), strings.TrimSpace(substituteTimestamps(stdout)))
+	assert.Equal(t, strings.TrimSpace(r.Config.StdOut), prepOutput(t, stdout))
 }
 
 func (r *Runner) VerifyStderr(t *testing.T, stderr string) {
-	assert.Equal(t, strings.TrimSpace(r.Config.Output[r.Format].StdErr), strings.TrimSpace(substituteTimestamps(stderr)))
+	assert.Equal(t, strings.TrimSpace(r.Config.StdErr), prepOutput(t, stderr))
+}
+
+func prepOutput(t *testing.T, s string) string {
+	return strings.TrimSpace(substituteTimestamps(removeStatusEvents(t, s)))
 }
 
 func (r *Runner) VerifyInventory(t *testing.T, name, namespace string) {
@@ -305,3 +179,32 @@ var timestampRegexp = regexp.MustCompile(`\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z`)
 func substituteTimestamps(text string) string {
 	return timestampRegexp.ReplaceAllString(text, "<TIMESTAMP>")
 }
+
+var statuses = []status.Status{
+	status.InProgressStatus,
+	status.CurrentStatus,
+	status.FailedStatus,
+	status.TerminatingStatus,
+	status.UnknownStatus,
+	status.NotFoundStatus,
+}
+
+func removeStatusEvents(t *testing.T, text string) string {
+	scanner := bufio.NewScanner(strings.NewReader(text))
+	var lines []string
+
+scan:
+	for scanner.Scan() {
+		line := scanner.Text()
+		for _, s := range statuses {
+			if strings.Contains(line, s.String()) {
+				continue scan
+			}
+		}
+		lines = append(lines, line)
+	}
+	if err := scanner.Err(); err != nil {
+		t.Fatalf("error scanning output: %v", err)
+	}
+	return strings.Join(lines, "\n")
+}
diff --git a/thirdparty/cmdconfig/commands/cmdeval/cmdeval_test.go b/thirdparty/cmdconfig/commands/cmdeval/cmdeval_test.go
index 03e007b1..bc2c0e61 100644
--- a/thirdparty/cmdconfig/commands/cmdeval/cmdeval_test.go
+++ b/thirdparty/cmdconfig/commands/cmdeval/cmdeval_test.go
@@ -219,7 +219,7 @@ apiVersion: v1
 			expectedStruct: &runfn.RunFns{
 				Path:                  dir,
 				ResultsDir:            "foo/",
-				ImagePullPolicy:       fnruntime.AlwaysPull,
+				ImagePullPolicy:       fnruntime.IfNotPresentPull,
 				Env:                   []string{},
 				ContinueOnEmptyResult: true,
 				Ctx:                   context.TODO(),
@@ -258,7 +258,7 @@ apiVersion: v1
 			path: dir,
 			expectedStruct: &runfn.RunFns{
 				Path:                  dir,
-				ImagePullPolicy:       fnruntime.AlwaysPull,
+				ImagePullPolicy:       fnruntime.IfNotPresentPull,
 				Env:                   []string{"FOO=BAR", "BAR"},
 				ContinueOnEmptyResult: true,
 				Ctx:                   context.TODO(),
@@ -283,7 +283,7 @@ apiVersion: v1
 			expectedStruct: &runfn.RunFns{
 				Path:                  dir,
 				AsCurrentUser:         true,
-				ImagePullPolicy:       fnruntime.AlwaysPull,
+				ImagePullPolicy:       fnruntime.IfNotPresentPull,
 				Env:                   []string{},
 				ContinueOnEmptyResult: true,
 				Ctx:                   context.TODO(),