Repository: GoogleContainerTools/skaffold
Pull request #8156 (2022-11-25 15:19:47)

chore: add log-streaming and cloud-run-proxy gcloud components 

<!-- Thank you for your contribution! -->

<!-- Include if applicable: -->
Fixes: #8058 <!-- tracking issues that this PR will close -->
**Related**: _Relevant tracking issues, for context_
**Merge before/after**: _Dependent or prerequisite PRs_

**Description**
 - add log-streaming and cloud-run-proxy gcloud components in relase dockerfile to support cloud run log streaming



codecov[bot]: # [Codecov](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools) Report
> Merging [#8156](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools) (92e32c2) into [main](https://codecov.io/gh/GoogleContainerTools/skaffold/commit/290280e8c6c19231e9ed5b5c027f2c01d22f75be?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools) (290280e) will **decrease** coverage by `4.27%`.
> The diff coverage is `53.26%`.

```diff
@@            Coverage Diff             @@
##             main    #8156      +/-   ##
==========================================
- Coverage   70.48%   66.21%   -4.28%     
==========================================
  Files         515      599      +84     
  Lines       23150    29346    +6196     
==========================================
+ Hits        16317    19431    +3114     
- Misses       5776     8461    +2685     
- Partials     1057     1454     +397     
```


| [Impacted Files](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools) | Coverage Δ | |
|---|---|---|
| [cmd/skaffold/app/cmd/completion.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvY29tcGxldGlvbi5nbw==) | `13.04% <0.00%> (-1.25%)` | :arrow_down: |
| [cmd/skaffold/app/cmd/credits/export.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvY3JlZGl0cy9leHBvcnQuZ28=) | `0.00% <0.00%> (ø)` | |
| [cmd/skaffold/app/cmd/deploy.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvZGVwbG95Lmdv) | `40.90% <0.00%> (-12.94%)` | :arrow_down: |
| [cmd/skaffold/app/cmd/test.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvdGVzdC5nbw==) | `44.44% <0.00%> (ø)` | |
| [cmd/skaffold/app/exitcode.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9leGl0Y29kZS5nbw==) | `100.00% <ø> (+6.66%)` | :arrow_up: |
| [cmd/skaffold/skaffold.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL3NrYWZmb2xkLmdv) | `0.00% <0.00%> (ø)` | |
| [cmd/skaffold/app/cmd/inspect\_tests.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvaW5zcGVjdF90ZXN0cy5nbw==) | `62.50% <14.28%> (-1.14%)` | :arrow_down: |
| [cmd/skaffold/app/cmd/render.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvcmVuZGVyLmdv) | `35.48% <18.18%> (-5.90%)` | :arrow_down: |
| [cmd/skaffold/app/cmd/lsp.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvbHNwLmdv) | `28.12% <28.12%> (ø)` | |
| [cmd/skaffold/app/cmd/run.go](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools#diff-Y21kL3NrYWZmb2xkL2FwcC9jbWQvcnVuLmdv) | `64.28% <33.33%> (-9.63%)` | :arrow_down: |
| ... and [394 more](https://codecov.io/gh/GoogleContainerTools/skaffold/pull/8156/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools) | |

:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=GoogleContainerTools)


Patch:
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
index 5947cec9f..d42c8b91c 100644
--- a/.github/workflows/codeql-analysis.yml
+++ b/.github/workflows/codeql-analysis.yml
@@ -17,7 +17,7 @@ concurrency:
 jobs:
   analyze:
     name: Analyze
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     permissions:
       actions: read
       contents: read
diff --git a/.github/workflows/draft-release.yml b/.github/workflows/draft-release.yml
index 32f930d22..7c1e372de 100644
--- a/.github/workflows/draft-release.yml
+++ b/.github/workflows/draft-release.yml
@@ -15,7 +15,7 @@ concurrency:
 jobs:
   check:
     name: Check if release commit
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     if: "startsWith(github.event.head_commit.message, 'release: ')"
     steps:
     - name: Apply release-matching regexp to commit message
@@ -46,7 +46,7 @@ jobs:
     if: needs.check.outputs.applicable == 'true'
     permissions:
       contents: write
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     steps:
     - name: Checkout code
       uses: actions/checkout@v3
diff --git a/.github/workflows/integration-linux.yml b/.github/workflows/integration-linux.yml
index 4af7e6ac8..b3825e6af 100644
--- a/.github/workflows/integration-linux.yml
+++ b/.github/workflows/integration-linux.yml
@@ -10,15 +10,17 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
+
   build:
     name: PR integration tests (linux)
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     strategy:
       matrix:
         kustomize_version: [3.5.4]
         ko_version: [0.4.0]
         kompose_version: [1.21.0]
-        gcloud_sdk_version: [335.0.0]
+        kpt_version: [1.0.0-beta.13]
+        gcloud_sdk_version: [410.0.0]
         container_structure_tests_version: [1.8.0]
         integration_test_partitions: [0, 1, 2, 3]
     steps:
@@ -77,12 +79,18 @@ jobs:
         wget -O kompose https://github.com/kubernetes/kompose/releases/download/v${{ matrix.kompose_version }}/kompose-linux-amd64 && chmod +x kompose
         sudo mv kompose /usr/local/bin/
 
+    - name: Install Kpt
+      if: ${{ env.NON_DOCS_FILES_CHANGED != 0 }}
+      run: |
+        wget -O kpt https://github.com/GoogleContainerTools/kpt/releases/download/v${{ matrix.kpt_version }}/kpt_linux_amd64 && chmod +x kpt
+        sudo mv kpt /usr/local/bin/
+
     - name: Install GCloud
       if: ${{ env.NON_DOCS_FILES_CHANGED != 0 }}
       run: |
         wget -O gcloud.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${{ matrix.gcloud_sdk_version }}-linux-x86_64.tar.gz
         tar -xvf gcloud.tar.gz -C ${HOME}/
-        CLOUDSDK_PYTHON="python2.7" ${HOME}/google-cloud-sdk/install.sh --usage-reporting=false --bash-completion=false --disable-installation-options
+        ${HOME}/google-cloud-sdk/install.sh --usage-reporting=false --bash-completion=false --disable-installation-options
         echo "${HOME}/google-cloud-sdk/bin" >> $GITHUB_PATH
 
     - name: Configure GCloud with Docker
diff --git a/.github/workflows/performance-comparison-label.yml b/.github/workflows/performance-comparison-label.yml
index 0977822af..069bcb58b 100644
--- a/.github/workflows/performance-comparison-label.yml
+++ b/.github/workflows/performance-comparison-label.yml
@@ -15,7 +15,7 @@ concurrency:
 jobs:
   build:
     if: ${{ github.event.label.name == 'help:run-comparisonstats' }}
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
 
     steps:
     - name: Remove run-performance-comparison label
diff --git a/.github/workflows/performance-comparison.yml b/.github/workflows/performance-comparison.yml
index 960938f42..79773a469 100644
--- a/.github/workflows/performance-comparison.yml
+++ b/.github/workflows/performance-comparison.yml
@@ -16,13 +16,13 @@ jobs:
   build:
     if: ${{ startsWith(github.event.comment.body, '/run-comparisonstats') }}
     name: Performance comparison (on comment)
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     strategy:
       matrix:
         kustomize_version: [3.5.4]
         ko_version: [0.4.0]
         kompose_version: [1.21.0]
-        gcloud_sdk_version: [335.0.0]
+        gcloud_sdk_version: [410.0.0]
         container_structure_tests_version: [1.8.0]
     steps:
     - name: "Check if user has write access"
@@ -68,7 +68,7 @@ jobs:
       run: |
         wget -O gcloud.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${{ matrix.gcloud_sdk_version }}-linux-x86_64.tar.gz
         tar -xvf gcloud.tar.gz -C ${HOME}/
-        CLOUDSDK_PYTHON="python2.7" ${HOME}/google-cloud-sdk/install.sh --usage-reporting=false --bash-completion=false --disable-installation-options
+        ${HOME}/google-cloud-sdk/install.sh --usage-reporting=false --bash-completion=false --disable-installation-options
         echo "${HOME}/google-cloud-sdk/bin" >> $GITHUB_PATH
 
     - name: Configure GCloud with Docker
diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml
index 5fb9b9781..d741c844f 100644
--- a/.github/workflows/scorecards.yml
+++ b/.github/workflows/scorecards.yml
@@ -13,7 +13,7 @@ permissions: read-all
 jobs:
   analysis:
     name: Scorecards analysis
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     permissions:
       # Needed to upload the results to code-scanning dashboard.
       security-events: write
diff --git a/.github/workflows/verify-examples.yml b/.github/workflows/verify-examples.yml
index 45dcbfd44..53ba32ad2 100644
--- a/.github/workflows/verify-examples.yml
+++ b/.github/workflows/verify-examples.yml
@@ -12,7 +12,7 @@ permissions: read-all
 jobs:
   dependabot:
     name: Verify changes to examples
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
 
     steps:
     - name: Check out code
diff --git a/deploy/skaffold/Dockerfile.deps b/deploy/skaffold/Dockerfile.deps
index 8a4319479..b71045b30 100644
--- a/deploy/skaffold/Dockerfile.deps
+++ b/deploy/skaffold/Dockerfile.deps
@@ -143,8 +143,10 @@ RUN /google-cloud-sdk/install.sh \
     --bash-completion=false \
     --disable-installation-options
 ENV PATH=$PATH:/google-cloud-sdk/bin
-RUN gcloud auth configure-docker && \
-    gcloud components install gke-gcloud-auth-plugin
+RUN gcloud auth configure-docker && gcloud components install --quiet \
+    gke-gcloud-auth-plugin \
+    cloud-run-proxy \
+    log-streaming
 
 FROM runtime_deps
 RUN apt-get update && apt-get install --no-install-recommends --no-install-suggests -y \
diff --git a/deploy/skaffold/Dockerfile.deps.lts b/deploy/skaffold/Dockerfile.deps.lts
index ecccac2ef..f5ecc3d00 100644
--- a/deploy/skaffold/Dockerfile.deps.lts
+++ b/deploy/skaffold/Dockerfile.deps.lts
@@ -87,8 +87,10 @@ RUN /google-cloud-sdk/install.sh \
     --bash-completion=false \
     --disable-installation-options
 ENV PATH=$PATH:/google-cloud-sdk/bin
-RUN gcloud auth configure-docker && \
-    gcloud components install gke-gcloud-auth-plugin
+RUN gcloud auth configure-docker && gcloud components install --quiet \
+    gke-gcloud-auth-plugin \
+    cloud-run-proxy \
+    log-streaming
 
 FROM runtime_deps
 RUN apt-get update && apt-get install --no-install-recommends --no-install-suggests -y \
diff --git a/deploy/skaffold/Dockerfile.deps.slim b/deploy/skaffold/Dockerfile.deps.slim
index ecccac2ef..f5ecc3d00 100644
--- a/deploy/skaffold/Dockerfile.deps.slim
+++ b/deploy/skaffold/Dockerfile.deps.slim
@@ -87,8 +87,10 @@ RUN /google-cloud-sdk/install.sh \
     --bash-completion=false \
     --disable-installation-options
 ENV PATH=$PATH:/google-cloud-sdk/bin
-RUN gcloud auth configure-docker && \
-    gcloud components install gke-gcloud-auth-plugin
+RUN gcloud auth configure-docker && gcloud components install --quiet \
+    gke-gcloud-auth-plugin \
+    cloud-run-proxy \
+    log-streaming
 
 FROM runtime_deps
 RUN apt-get update && apt-get install --no-install-recommends --no-install-suggests -y \
diff --git a/docs-v2/content/en/docs/tutorials/skaffold-resource-selector.md b/docs-v2/content/en/docs/tutorials/skaffold-resource-selector.md
index 51757f070..57e423c94 100644
--- a/docs-v2/content/en/docs/tutorials/skaffold-resource-selector.md
+++ b/docs-v2/content/en/docs/tutorials/skaffold-resource-selector.md
@@ -12,7 +12,7 @@ Common Use Cases This Page Helps Resolve:
 * Users who are seeing issues with skaffold's default resource field overwriting for a given resource - eg: skaffold errors as it tries to mutate immutable config on re-deployment
 
 Currently skaffold modifies the manifests it renders and deploys for the following functionality:
-- status checking - done by mutating the manifest/K8s-Object by adding a label - skaffold/dev/run-id.  Skaffold uses this run-id to identify ... (TODO add doc link to run-id explanation)
+- status checking - done by mutating the manifest/K8s-Object by adding a label - skaffold/dev/run-id.  Skaffold uses this run-id to identify the deployments Skaffold manages with it's status checking.
 - image label overwriting - done by mutating the manifest/K8s-Object by substituting the `image:$ORIGINAL_IMAGE_TAG` value(s) in a manifest with `image:$RECENT_SKAFFOLD_BUILT_IMAGE`
 
 
@@ -47,7 +47,7 @@ _* *.image (changes `image:` value to be the skaffold built image ONLY IF skaffo
 
 
 The GroupKind's that Skaffold manages (via resource field overwriting) are user configurable via the `resourceSelector:` top level configuration.  The `resourceSelector` configuration allows users to modify and extend which resources and what fields of those resources skaffold modifies.  Currently skaffold only supports `label:` and `.metadata.labels` related modifications.
-(TODO add `resourceSelector` schema overview and allowable inputs)
+
 `resourceSelector` spec (from `pkg/skaffold/schema/latest/config.go`)
 ```
 // ResourceSelector describes user defined filters describing how skaffold should treat objects/fields during rendering.
diff --git a/examples/bazel/.bazelrc b/examples/bazel/.bazelrc
deleted file mode 100644
index 680549907..000000000
--- a/examples/bazel/.bazelrc
+++ /dev/null
@@ -1,8 +0,0 @@
-# The following flags are set to test use of new features for python toolchains
-# These flags will only work with Bazel 0.25.0 or above.
-build --incompatible_use_python_toolchains
-build --host_force_python=PY2
-test --incompatible_use_python_toolchains
-test --host_force_python=PY2
-run --incompatible_use_python_toolchains
-run --host_force_python=PY2
diff --git a/examples/bazel/WORKSPACE b/examples/bazel/WORKSPACE
index 5637093ed..3b5a76e11 100644
--- a/examples/bazel/WORKSPACE
+++ b/examples/bazel/WORKSPACE
@@ -4,9 +4,8 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 
 http_archive(
     name = "io_bazel_rules_docker",
-    sha256 = "4521794f0fba2e20f3bf15846ab5e01d5332e587e9ce81629c7f96c793bb7036",
-    strip_prefix = "rules_docker-0.14.4",
-    urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.14.4/rules_docker-v0.14.4.tar.gz"],
+    sha256 = "b1e80761a8a8243d03ebca8845e9cc1ba6c82ce7c5179ce2b295cd36f7e394bf",
+    urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.25.0/rules_docker-v0.25.0.tar.gz"],
 )
 
 http_archive(
@@ -18,9 +17,10 @@ http_archive(
     ],
 )
 
-load("@io_bazel_rules_go//go:deps.bzl", "go_rules_dependencies", "go_register_toolchains")
+load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
 
 go_rules_dependencies()
+
 go_register_toolchains(
     go_version = "1.14.4",
 )
@@ -39,13 +39,9 @@ load(
 
 container_deps()
 
-load("@io_bazel_rules_docker//repositories:pip_repositories.bzl", "pip_deps")
-
-pip_deps()
-
 load(
     "@io_bazel_rules_docker//go:image.bzl",
     _go_image_repos = "repositories",
 )
 
-_go_image_repos()
\ No newline at end of file
+_go_image_repos()
diff --git a/integration/examples/bazel/.bazelrc b/integration/examples/bazel/.bazelrc
deleted file mode 100644
index 680549907..000000000
--- a/integration/examples/bazel/.bazelrc
+++ /dev/null
@@ -1,8 +0,0 @@
-# The following flags are set to test use of new features for python toolchains
-# These flags will only work with Bazel 0.25.0 or above.
-build --incompatible_use_python_toolchains
-build --host_force_python=PY2
-test --incompatible_use_python_toolchains
-test --host_force_python=PY2
-run --incompatible_use_python_toolchains
-run --host_force_python=PY2
diff --git a/integration/examples/bazel/WORKSPACE b/integration/examples/bazel/WORKSPACE
index 5637093ed..3b5a76e11 100644
--- a/integration/examples/bazel/WORKSPACE
+++ b/integration/examples/bazel/WORKSPACE
@@ -4,9 +4,8 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 
 http_archive(
     name = "io_bazel_rules_docker",
-    sha256 = "4521794f0fba2e20f3bf15846ab5e01d5332e587e9ce81629c7f96c793bb7036",
-    strip_prefix = "rules_docker-0.14.4",
-    urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.14.4/rules_docker-v0.14.4.tar.gz"],
+    sha256 = "b1e80761a8a8243d03ebca8845e9cc1ba6c82ce7c5179ce2b295cd36f7e394bf",
+    urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.25.0/rules_docker-v0.25.0.tar.gz"],
 )
 
 http_archive(
@@ -18,9 +17,10 @@ http_archive(
     ],
 )
 
-load("@io_bazel_rules_go//go:deps.bzl", "go_rules_dependencies", "go_register_toolchains")
+load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
 
 go_rules_dependencies()
+
 go_register_toolchains(
     go_version = "1.14.4",
 )
@@ -39,13 +39,9 @@ load(
 
 container_deps()
 
-load("@io_bazel_rules_docker//repositories:pip_repositories.bzl", "pip_deps")
-
-pip_deps()
-
 load(
     "@io_bazel_rules_docker//go:image.bzl",
     _go_image_repos = "repositories",
 )
 
-_go_image_repos()
\ No newline at end of file
+_go_image_repos()
diff --git a/pkg/skaffold/deploy/util/util.go b/pkg/skaffold/deploy/util/util.go
index d55ee9db6..b69636154 100644
--- a/pkg/skaffold/deploy/util/util.go
+++ b/pkg/skaffold/deploy/util/util.go
@@ -18,7 +18,6 @@ package util
 
 import (
 	"context"
-	"errors"
 	"fmt"
 	"io"
 	"os"
@@ -38,7 +37,6 @@ import (
 	"github.com/GoogleContainerTools/skaffold/pkg/skaffold/kubernetes"
 	"github.com/GoogleContainerTools/skaffold/pkg/skaffold/kubernetes/manifest"
 	"github.com/GoogleContainerTools/skaffold/pkg/skaffold/output/log"
-	"github.com/GoogleContainerTools/skaffold/pkg/skaffold/util"
 	"github.com/GoogleContainerTools/skaffold/pkg/skaffold/util/stringset"
 )
 
@@ -106,10 +104,15 @@ func ConsolidateNamespaces(original, new []string) []string {
 }
 
 // GetHydrationDir points to the directory where the manifest rendering happens. By default, it is set to "<WORKDIR>/.kpt-pipeline".
-func GetHydrationDir(opts config.SkaffoldOptions, workingDir string, promptIfNeeded bool) (string, error) {
+func GetHydrationDir(opts config.SkaffoldOptions, workingDir string, promptIfNeeded bool, isKptRendererOrDeployerUsed bool) (string, error) {
 	var hydratedDir string
 	var err error
 
+	if !isKptRendererOrDeployerUsed {
+		log.Entry(context.TODO()).Info("no kpt renderer or deployer found, skipping hydrated-dir creation")
+		return "", nil
+	}
+
 	if opts.HydrationDir == constants.DefaultHydrationDir {
 		hydratedDir = filepath.Join(workingDir, constants.DefaultHydrationDir)
 		promptIfNeeded = false
@@ -164,37 +167,6 @@ func GroupVersionResource(disco discovery.DiscoveryInterface, gvk schema.GroupVe
 	return false, schema.GroupVersionResource{}, fmt.Errorf("could not find resource for %s", gvk.String())
 }
 
-func GetManifestsFromHydrationDir(ctx context.Context, opts config.SkaffoldOptions) (manifest.ManifestList, error) {
-	workDir, err := util.RealWorkDir()
-	if err != nil {
-		return nil, fmt.Errorf("getting working directory: %w", err)
-	}
-	hydrationDir, err := GetHydrationDir(config.SkaffoldOptions{HydrationDir: opts.HydrationDir, AssumeYes: opts.AssumeYes},
-		workDir,
-		true)
-	if err != nil {
-		return nil, fmt.Errorf("getting render output path: %w", err)
-	}
-	// TODO(aaron-prindle) verify that using manifests.yaml here will work in all cases
-	hydratedManifestPath := filepath.Join(hydrationDir, "manifests.yaml")
-	if _, err := os.Stat(hydratedManifestPath); errors.Is(err, os.ErrNotExist) {
-		log.Entry(ctx).Warn(fmt.Errorf("unable to find manifests, attempted location %s: %w", hydratedManifestPath, err))
-		return nil, nil
-	}
-
-	f, err := os.Open(hydratedManifestPath)
-	if err != nil {
-		return nil, fmt.Errorf("opening hydrated manifest at %s: %w", hydratedManifestPath, err)
-	}
-	defer f.Close()
-
-	manifests, err := manifest.Load(f)
-	if err != nil {
-		return nil, fmt.Errorf("parsing manifests file into manifest list object: %w", err)
-	}
-	return manifests, nil
-}
-
 func GetManifestsFromHydratedManifests(ctx context.Context, hydratedManifests []string) (manifest.ManifestList, error) {
 	var manifests manifest.ManifestList
 	for _, path := range hydratedManifests {
diff --git a/pkg/skaffold/runner/new.go b/pkg/skaffold/runner/new.go
index 274c50820..1d4d77a50 100644
--- a/pkg/skaffold/runner/new.go
+++ b/pkg/skaffold/runner/new.go
@@ -74,7 +74,7 @@ func NewForConfig(ctx context.Context, runCtx *runcontext.RunContext) (*Skaffold
 
 	var deployer deploy.Deployer
 
-	hydrationDir, err := util.GetHydrationDir(runCtx.Opts, runCtx.WorkingDir, true)
+	hydrationDir, err := util.GetHydrationDir(runCtx.Opts, runCtx.WorkingDir, true, isKptRendererOrDeployerUsed(runCtx.Pipelines))
 
 	if err != nil {
 		return nil, fmt.Errorf("getting render output path: %w", err)
@@ -253,3 +253,21 @@ func getTester(ctx context.Context, cfg test.Config, isLocalImage func(imageName
 
 	return tester, nil
 }
+
+func isKptRendererOrDeployerUsed(pipelines runcontext.Pipelines) bool {
+	for _, configName := range pipelines.AllOrderedConfigNames() {
+		pipeline := pipelines.GetForConfigName(configName)
+		renderConfig := pipeline.Render
+		deployConfig := pipeline.Deploy
+
+		if renderConfig.Validate != nil || renderConfig.Transform != nil || renderConfig.Kpt != nil {
+			return true
+		}
+
+		if deployConfig.KptDeploy != nil {
+			return true
+		}
+	}
+
+	return false
+}
diff --git a/pkg/skaffold/runner/runcontext/context.go b/pkg/skaffold/runner/runcontext/context.go
index a2d3a55a6..4eab76015 100644
--- a/pkg/skaffold/runner/runcontext/context.go
+++ b/pkg/skaffold/runner/runcontext/context.go
@@ -261,6 +261,18 @@ func (rc *RunContext) GetNamespace() string {
 	if rc.Opts.Namespace != "" {
 		return rc.Opts.Namespace
 	}
+	var defaultNamespace string
+	for _, p := range rc.GetPipelines() {
+		if p.Deploy.KubectlDeploy != nil && p.Deploy.KubectlDeploy.DefaultNamespace != nil {
+			if defaultNamespace != "" {
+				log.Entry(context.TODO()).Warn("multiple deploy.kubectl.defaultNamespace values set, only last pipeline's value will be used")
+			}
+			defaultNamespace = *p.Deploy.KubectlDeploy.DefaultNamespace
+		}
+	}
+	if defaultNamespace != "" {
+		return defaultNamespace
+	}
 	b, err := (&util.Commander{}).RunCmdOut(context.Background(), exec.Command("kubectl", "config", "view", "--minify", "-o", "jsonpath='{..namespace}'"))
 	if err != nil {
 		return rc.Opts.Namespace
diff --git a/pkg/skaffold/schema/util/util.go b/pkg/skaffold/schema/util/util.go
index 765478876..153733670 100644
--- a/pkg/skaffold/schema/util/util.go
+++ b/pkg/skaffold/schema/util/util.go
@@ -92,45 +92,34 @@ func (m *FlatMap) UnmarshalYAML(unmarshal func(interface{}) error) error {
 		return err
 	}
 	result := make(map[string]string)
-	if err := buildFlatMap(obj, result, ""); err != nil {
-		return err
-	}
+	buildFlatMap(obj, result, "")
 	*m = result
 	return nil
 }
 
-func buildFlatMap(obj map[string]interface{}, result map[string]string, currK string) (err error) {
-	var prevK string
-	for k, v := range obj {
-		prevK = currK
-		if currK == "" {
-			currK = fmt.Sprintf("%v", k)
-		} else {
-			currK = fmt.Sprintf("%v.%v", currK, k)
-		}
+func buildFlatList(list []interface{}, result map[string]string, parentK string) {
+	for idx, item := range list {
+		currK := fmt.Sprintf("%v[%d]", parentK, idx)
+		processItem(item, result, currK)
+	}
+}
 
-		switch v := v.(type) {
-		case map[string]interface{}:
-			if err = buildFlatMap(v, result, currK); err != nil {
-				return
-			}
-		case []interface{}:
-			for idx, i := range v {
-				if m, ok := i.(map[string]interface{}); ok {
-					currIdx := fmt.Sprintf("%v[%d]", currK, idx)
-					if err = buildFlatMap(m, result, currIdx); err != nil {
-						return
-					}
-				}
-			}
-		case string:
-			result[currK] = v
-		default:
-			result[currK] = fmt.Sprintf("%v", v)
-		}
-		currK = prevK
+func buildFlatMap(obj map[string]interface{}, result map[string]string, parentK string) {
+	for key, item := range obj {
+		currK := fmt.Sprintf("%v%v", parentK, key)
+		processItem(item, result, currK)
+	}
+}
+
+func processItem(item interface{}, result map[string]string, currK string) {
+	switch v := item.(type) {
+	case map[string]interface{}:
+		buildFlatMap(v, result, fmt.Sprintf("%v.", currK))
+	case []interface{}:
+		buildFlatList(v, result, currK)
+	default:
+		result[currK] = fmt.Sprintf("%v", v)
 	}
-	return err
 }
 
 func marshalInlineYaml(in interface{}) ([]byte, error) {


Test patch:
diff --git a/.github/workflows/linters-checks-and-unit-tests-linux.yml b/.github/workflows/linters-checks-and-unit-tests-linux.yml
index 02983fa7f..70f504224 100644
--- a/.github/workflows/linters-checks-and-unit-tests-linux.yml
+++ b/.github/workflows/linters-checks-and-unit-tests-linux.yml
@@ -13,7 +13,7 @@ jobs:
 
   build:
     name: PR linters, checks, and unit tests (linux)
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
     steps:
     - name: Set up Go
       uses: actions/setup-go@v3
diff --git a/integration/render_test.go b/integration/render_test.go
index 4549f277c..1394bfa07 100644
--- a/integration/render_test.go
+++ b/integration/render_test.go
@@ -22,6 +22,7 @@ import (
 	"fmt"
 	"os"
 	"path"
+	"path/filepath"
 	"regexp"
 	"testing"
 
@@ -705,3 +706,134 @@ spec:
 		})
 	}
 }
+
+func TestRenderHydrationDirCreation(t *testing.T) {
+	MarkIntegrationTest(t, CanRunWithoutGcp)
+	const hydrationDir = "hydration-dir"
+
+	tests := []struct {
+		description              string
+		shouldCreateHydrationDir bool
+		config                   string
+		manifest                 string
+	}{
+		{
+			description:              "project with kpt renderer should create hydration dir",
+			shouldCreateHydrationDir: true,
+			config: `
+apiVersion: skaffold/v4beta1
+kind: Config
+
+build:
+  artifacts: []
+manifests:
+  kpt: []
+`,
+			manifest: `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: getting-started
+spec:
+  containers:
+    - name: getting-started
+      image: skaffold-example`,
+		},
+		{
+			description:              "project with kpt deployer should create hydration dir",
+			shouldCreateHydrationDir: true,
+			config: `
+apiVersion: skaffold/v4beta1
+kind: Config
+
+build:
+  artifacts: []
+manifests:
+  kpt: []
+deploy:
+  kpt: {}
+`,
+			manifest: `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: getting-started
+spec:
+  containers:
+    - name: getting-started
+      image: skaffold-example`,
+		},
+		{
+			description:              "project with rawYaml and transform should create hydration dir (uses kpt renderer)",
+			shouldCreateHydrationDir: true,
+			config: `
+apiVersion: skaffold/v4beta1
+kind: Config
+
+build:
+  artifacts: []
+manifests:
+  rawYaml:
+    - k8s/k8s-pod.yaml
+  transform:
+    - name: set-annotations
+      configMap:
+        - "author:fake-author"
+`,
+			manifest: `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: getting-started
+spec:
+  containers:
+    - name: getting-started
+      image: skaffold-example`,
+		},
+		{
+			description:              "project without kpt should not create hydration dir",
+			shouldCreateHydrationDir: false,
+			config: `
+apiVersion: skaffold/v4beta1
+kind: Config
+
+build:
+  artifacts: []
+manifests:
+  rawYaml:
+    - k8s/k8s-pod.yaml
+`,
+			manifest: `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: getting-started
+spec:
+  containers:
+  - name: getting-started
+    image: skaffold-example`,
+		},
+	}
+
+	for _, test := range tests {
+		testutil.Run(t, test.description, func(t *testutil.T) {
+			tmpDir := t.NewTempDir()
+			tmpDir.Mkdir("k8s")
+			tmpDir.Write("skaffold.yaml", test.config)
+			tmpDir.Write("k8s/k8s-pod.yaml", test.manifest)
+			tmpDir.Chdir()
+
+			args := []string{"--hydration-dir", hydrationDir}
+
+			skaffold.Render(args...).RunOrFail(t.T)
+
+			_, err := os.Stat(filepath.Join(tmpDir.Root(), hydrationDir))
+
+			if test.shouldCreateHydrationDir {
+				t.CheckFalse(os.IsNotExist(err))
+			} else {
+				t.CheckTrue(os.IsNotExist(err))
+			}
+		})
+	}
+}
diff --git a/pkg/skaffold/deploy/util/util_test.go b/pkg/skaffold/deploy/util/util_test.go
index 191a66214..f9573b768 100644
--- a/pkg/skaffold/deploy/util/util_test.go
+++ b/pkg/skaffold/deploy/util/util_test.go
@@ -112,7 +112,7 @@ func TestGetHydrationDir_Default(t *testing.T) {
 		tmpDir.Chdir()
 		actual, err := GetHydrationDir(
 			config.SkaffoldOptions{HydrationDir: constants.DefaultHydrationDir, AssumeYes: true},
-			tmpDir.Root(), false)
+			tmpDir.Root(), false, true)
 		t.CheckNoError(err)
 		t.CheckDeepEqual(filepath.Join(tmpDir.Root(), ".kpt-pipeline"), actual)
 	})
@@ -124,7 +124,7 @@ func TestGetHydrationDir_CustomHydrationDir(t *testing.T) {
 		tmpDir.Chdir()
 		expected := filepath.Join(tmpDir.Root(), "test-hydration")
 		actual, err := GetHydrationDir(
-			config.SkaffoldOptions{HydrationDir: expected, AssumeYes: true}, "", false)
+			config.SkaffoldOptions{HydrationDir: expected, AssumeYes: true}, "", false, true)
 		t.CheckNoError(err)
 		t.CheckDeepEqual(expected, actual)
 		_, err = os.Stat(actual)
diff --git a/pkg/skaffold/schema/util/util_test.go b/pkg/skaffold/schema/util/util_test.go
index 9a4a41623..269ab39b1 100644
--- a/pkg/skaffold/schema/util/util_test.go
+++ b/pkg/skaffold/schema/util/util_test.go
@@ -174,3 +174,71 @@ values.val3[0].val9: foo4
   values.val3[0].val9: foo4
 `, string(out))
 }
+
+func TestFlatMap_UnmarshalYAMLNestedArrays(t *testing.T) {
+	inputYaml := `
+name: nameValue
+configs:
+  name: configName
+  defaults:
+    - prop1: bar1
+      prop2: foo1
+    - prop1: bar2
+      prop2: foo2
+  mixlist:
+    - prop1: bar3
+      prop2:
+       - foo3
+    - element2
+  matrix:
+    - - i1
+      - i2
+    - - j1
+      - j2
+`
+
+	expected := `name: nameValue
+configs.name: configName
+configs.defaults[0].prop1: bar1
+configs.defaults[0].prop2: foo1
+configs.defaults[1].prop1: bar2
+configs.defaults[1].prop2: foo2
+configs.mixlist[0].prop1: bar3
+configs.mixlist[0].prop2[0]: foo3
+configs.mixlist[1]: element2
+configs.matrix[0][0]: i1
+configs.matrix[0][1]: i2
+configs.matrix[1][0]: j1
+configs.matrix[1][1]: j2
+`
+	inputFlatmap := &FlatMap{}
+	expectedFlatmap := &FlatMap{}
+
+	err := yaml.Unmarshal([]byte(inputYaml), &inputFlatmap)
+	testutil.CheckError(t, false, err)
+
+	err = yaml.Unmarshal([]byte(expected), &expectedFlatmap)
+	testutil.CheckError(t, false, err)
+
+	testutil.CheckDeepEqual(t, *inputFlatmap, *expectedFlatmap)
+
+	out, err := yaml.Marshal(struct {
+		M *FlatMap `yaml:"value,omitempty"`
+	}{inputFlatmap})
+
+	testutil.CheckErrorAndDeepEqual(t, false, err, `value:
+  configs.defaults[0].prop1: bar1
+  configs.defaults[0].prop2: foo1
+  configs.defaults[1].prop1: bar2
+  configs.defaults[1].prop2: foo2
+  configs.matrix[0][0]: i1
+  configs.matrix[0][1]: i2
+  configs.matrix[1][0]: j1
+  configs.matrix[1][1]: j2
+  configs.mixlist[0].prop1: bar3
+  configs.mixlist[0].prop2[0]: foo3
+  configs.mixlist[1]: element2
+  configs.name: configName
+  name: nameValue
+`, string(out))
+}