Repository: linkerd/linkerd2
Pull request #4034 (2020-02-10 16:26:44)

Ensure we get the correct type out of Informer Deletion events
It turns out that when we define our `DeleteFunc` handler it is possible in certain unfortunate situations to get a type different than what we were expecting (we want a *Service, we get `DeletedFinalStateUnknown`). If that happens we will probably fail with some cast exception. This Pr here ensures we consider these cases and cast/extract types in safer manner. 

 Wrt to the `DeletedFinalStateUnknown`, it  is placed into a DeltaFIFO in the case where an object was deleted but the watch deletion event was missed. In this case we don't know the final "resting" state of the object, so there's a chance the included `Obj` is stale. 

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

zaharidichev: @alpeb. Are you asking whether we should simply have the logic of unpacking? I guess what I was trying to achieve here is to contain the checking in one method only. That allows me to ensure that if tomorrow someone on the k8s team decides to introduce yet another wrapper to express some state that I never though of, our code would not panic with an unsafe cast but would simply skip casting and log an error. Does that make sense

Patch:
diff --git a/.github/workflows/cloud_integration.yml b/.github/workflows/cloud_integration.yml
new file mode 100644
index 000000000..6e3756b01
--- /dev/null
+++ b/.github/workflows/cloud_integration.yml
@@ -0,0 +1,134 @@
+name: Cloud integration
+on:
+  push:
+    branches:
+    - master
+    tags:
+    - "*"
+jobs:
+  docker_build:
+    name: Docker build
+    runs-on: ubuntu-18.04
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Set environment variables from scripts
+      run: |
+        . bin/_tag.sh
+        echo ::set-env name=TAG::$(CI_FORCE_CLEAN=1 bin/root-tag)
+    - name: Setup SSH config for Packet
+      run: |
+        mkdir -p ~/.ssh/
+        touch ~/.ssh/id && chmod 600 ~/.ssh/id
+        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
+        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
+        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
+        ssh linkerd-docker docker version
+    - name: Build docker images
+      env:
+        DOCKER_HOST: ssh://linkerd-docker
+        DOCKER_TRACE: 1
+      run: |
+        export PATH="`pwd`/bin:$PATH"
+        bin/docker-build
+  docker_push:
+    name: Docker push
+    runs-on: ubuntu-18.04
+    needs: [docker_build]
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Set environment variables from scripts
+      run: |
+        . bin/_tag.sh
+        echo ::set-env name=TAG::$(CI_FORCE_CLEAN=1 bin/root-tag)
+    - name: Configure gcloud
+      uses: linkerd/linkerd2-action-gcloud@v1.0.0
+      with:
+        cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
+        gcp_project: ${{ secrets.GCP_PROJECT }}
+        gcp_zone: ${{ secrets.GCP_ZONE }}
+    - name: Docker SSH setup
+      run: |
+        mkdir -p ~/.ssh/
+        touch ~/.ssh/id && chmod 600 ~/.ssh/id
+        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
+        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
+        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
+        ssh linkerd-docker docker version
+    - name: Push docker images to registry
+      env:
+        DOCKER_HOST: ssh://linkerd-docker
+      run: |
+        export PATH="`pwd`/bin:$PATH"
+        bin/docker-push-deps
+        bin/docker-push $TAG
+        bin/docker-retag-all $TAG master
+        bin/docker-push master
+  cloud_integration_tests:
+    name: Cloud integration tests
+    runs-on: ubuntu-18.04
+    needs: [docker_push]
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Install linkerd CLI
+      id: install_cli
+      run: |
+        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
+        image="gcr.io/linkerd-io/cli-bin:$TAG"
+        id=$(bin/docker create $image)
+        bin/docker cp "$id:/out/linkerd-linux" "$HOME/.linkerd"
+        $HOME/.linkerd version --client
+        # validate CLI version matches the repo
+        [[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
+        echo "Installed Linkerd CLI version: $TAG"
+        echo "::set-output name=tag::$TAG"
+    - name: Create GKE cluster
+      uses: linkerd/linkerd2-action-gcloud@v1.0.0
+      with:
+        cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
+        gcp_project: ${{ secrets.GCP_PROJECT }}
+        gcp_zone: ${{ secrets.GCP_ZONE }}
+        create: true
+        name: testing-${{ steps.install_cli.outputs.tag }}-${{ github.run_id }}
+    - name: Run integration tests
+      env:
+        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
+      run: |
+        export PATH="`pwd`/bin:$PATH"
+        echo "$GITCOOKIE_SH" | bash
+        version="$($HOME/.linkerd version --client --short | tr -cd '[:alnum:]-')"
+        bin/test-run $HOME/.linkerd linkerd-$version
+    - name: CNI tests
+      run: |
+        export TAG="$($HOME/.linkerd version --client --short)"
+        go test -cover -race -v -mod=readonly ./cni-plugin/test -integration-tests
+  chart_deploy:
+    name: Helm chart deploy
+    if: startsWith(github.ref, 'refs/tags')
+    runs-on: ubuntu-18.04
+    needs: [cloud_integration_tests]
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Configure gsutils
+      uses: linkerd/linkerd2-action-gcloud@v1.0.0
+      with:
+        cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
+        gcp_project: ${{ secrets.GCP_PROJECT }}
+        gcp_zone: ${{ secrets.GCP_ZONE }}
+    - name: Edge Helm chart creation and upload
+      if: startsWith(github.ref, 'refs/tags/edge')
+      run: |
+        mkdir -p target/helm
+        gsutil cp gs://helm.linkerd.io/edge/index.yaml target/helm/index-pre.yaml
+        bin/helm-build package
+        gsutil rsync target/helm gs://helm.linkerd.io/edge
+    - name: Stable Helm chart creation and upload
+      if: startsWith(github.ref, 'refs/tags/stable')
+      run: |
+        mkdir -p target/helm
+        gsutil cp gs://helm.linkerd.io/stable/index.yaml target/helm/index-pre.yaml
+        bin/helm-build package
+        gsutil rsync target/helm gs://helm.linkerd.io/stable
diff --git a/.github/workflows/kind_integration.yml b/.github/workflows/kind_integration.yml
new file mode 100644
index 000000000..dfb07ce1a
--- /dev/null
+++ b/.github/workflows/kind_integration.yml
@@ -0,0 +1,176 @@
+name: KinD integration
+on:
+  pull_request: {}
+  push:
+    branches:
+    - master
+    tags:
+    - "*"
+jobs:
+  docker_build:
+    name: Docker build
+    runs-on: ubuntu-18.04
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Set environment variables from scripts
+      run: |
+        . bin/_tag.sh
+        echo ::set-env name=TAG::$(CI_FORCE_CLEAN=1 bin/root-tag)
+
+        . bin/_docker.sh
+        echo ::set-env name=DOCKER_REGISTRY::$DOCKER_REGISTRY
+    - name: Setup SSH config for Packet
+      if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
+      run: |
+        mkdir -p ~/.ssh/
+        touch ~/.ssh/id && chmod 600 ~/.ssh/id
+        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
+        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
+        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
+        ssh linkerd-docker docker version
+        echo ::set-env name=DOCKER_HOST::ssh://linkerd-docker
+    - name: Build docker images
+      env:
+        DOCKER_TRACE: 1
+      run: |
+        export PATH="`pwd`/bin:$PATH"
+        bin/docker-build
+    - name: Create artifact with CLI and image archives (Forked repositories)
+      if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
+      env:
+        ARCHIVES: /home/runner/archives
+      run: |
+        mkdir -p $ARCHIVES
+
+        for image in proxy controller web cni-plugin debug cli-bin grafana; do
+          docker save "$DOCKER_REGISTRY/$image:$TAG" > $ARCHIVES/$image.tar || tee save_fail &
+        done
+
+        # Wait for `docker save` background processes to complete. Exit early
+        # if any job failed.
+        wait < <(jobs -p)
+        test -f save_fail && exit 1 || true
+    # `with.path` values do not support environment variables yet, so an
+    # absolute path is used here.
+    #
+    # https://github.com/actions/upload-artifact/issues/8
+    - name: Upload artifact (Forked repositories)
+      if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
+      uses: actions/upload-artifact@v1
+      with:
+        name: image-archives
+        path: /home/runner/archives
+  kind_integration_tests:
+    strategy:
+      matrix:
+        integration_test: [deep, upgrade, helm, helm_upgrade, custom_domain, external_issuer]
+    needs: [docker_build]
+    name: Integration tests (${{ matrix.integration_test }})
+    runs-on: ubuntu-18.04
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Try to load cached Go modules
+      uses: actions/cache@v1
+      with:
+        path: ~/go/pkg/mod
+        key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
+        restore-keys: |
+          ${{ runner.os }}-go-
+    - name: Set environment variables from scripts
+      run: |
+        . bin/_tag.sh
+        echo ::set-env name=TAG::$(CI_FORCE_CLEAN=1 bin/root-tag)
+
+        . bin/_docker.sh
+        echo ::set-env name=DOCKER_REGISTRY::$DOCKER_REGISTRY
+    - name: Setup SSH config for Packet
+      if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
+      run: |
+        mkdir -p ~/.ssh/
+        touch ~/.ssh/id && chmod 600 ~/.ssh/id
+        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
+        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
+        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
+    - name: Download image archives (Forked repositories)
+      if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
+      uses: actions/download-artifact@v1
+      with:
+        name: image-archives
+    - name: Load cli-bin image into local docker images
+      if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
+      run: |
+        # `docker load` only accepts input from STDIN, so pipe the image
+        # archive into the command.
+        #
+        # In order to pipe the image archive, set `DOCKER_HOST` for a single
+        # command and `docker save` the CLI image from the Packet host.
+        DOCKER_HOST=ssh://linkerd-docker docker save "$DOCKER_REGISTRY/cli-bin:$TAG" | docker load
+    - name: Load cli-bin image into local docker images (Forked repositories)
+      if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
+      run: docker load < image-archives/cli-bin.tar
+    - name: Install CLI
+      run: |
+        # Copy the CLI out of the local cli-bin container.
+        container_id=$(docker create "$DOCKER_REGISTRY/cli-bin:$TAG")
+        docker cp $container_id:/out/linkerd-linux $HOME/.linkerd
+
+        # Validate the CLI version matches the current build tag.
+        [[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
+    - name: Setup default KinD cluster
+      if: matrix.integration_test != 'custom_domain'
+      uses: engineerd/setup-kind@v0.3.0
+      with:
+        version: "v0.6.1"
+    - name: Setup custom_domain KinD cluster
+      if: matrix.integration_test == 'custom_domain'
+      uses: engineerd/setup-kind@v0.3.0
+      with:
+        config: test/testdata/custom_cluster_domain_config.yaml
+        version: "v0.6.1"
+    - name: Load image archives into the local KinD cluster
+      if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
+      env:
+        PROXY_INIT_IMAGE_NAME: gcr.io/linkerd-io/proxy-init:v1.3.1
+        PROMETHEUS_IMAGE_NAME: prom/prometheus:v2.15.2
+      run: |
+        # For each container, load the image archive into the KinD cluster.
+        #
+        # `kind load` cannot take input from STDIN, so `<(command)` syntax is
+        # used to load the output into the KinD cluster. Set `DOCKER_HOST` for
+        # a single command, and `docker save` the container from the Packet
+        # host.
+        for image in proxy controller web cni-plugin debug grafana; do
+          kind load image-archive <(DOCKER_HOST=ssh://linkerd-docker docker save "$DOCKER_REGISTRY/$image:$TAG") || tee load_fail &
+        done
+
+        # Wait for `kind load` background processes to complete. Exit early if
+        # any job failed.
+        wait < <(jobs -p)
+        test -f load_fail && exit 1 || true
+
+        # Load proxy-init and prometheus images into KinD while it is
+        # available. Allow these commands to fail since they will be cached
+        # for the next run.
+        kind load image-archive <(DOCKER_HOST=ssh://linkerd-docker docker save $PROXY_INIT_IMAGE_NAME) 2>&1 || true
+        kind load image-archive <(DOCKER_HOST=ssh://linkerd-docker docker save $PROMETHEUS_IMAGE_NAME) 2>&1 || true
+    - name: Load image archives into the local KinD cluster (Forked repositories)
+      if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
+      run: |
+        for image in proxy controller web cni-plugin debug grafana; do
+          kind load image-archive image-archives/$image.tar || tee load_fail &
+        done
+
+        # Wait for `kind load` background processes to complete. Exit early if
+        # any job failed.
+        wait < <(jobs -p)
+        test -f load_fail && exit 1 || true
+    - name: Run integration tests
+      run: |
+        # Export `init_test_run` and `*_integration_tests` into the
+        # environment.
+        . bin/_test-run.sh
+
+        init_test_run $HOME/.linkerd
+        ${{ matrix.integration_test }}_integration_tests
diff --git a/.github/workflows/static_checks.yml b/.github/workflows/static_checks.yml
new file mode 100644
index 000000000..d69dd9f5e
--- /dev/null
+++ b/.github/workflows/static_checks.yml
@@ -0,0 +1,59 @@
+name: Static checks
+on:
+  pull_request: {}
+  push:
+    branches:
+    - master
+    tags:
+    - "*"
+jobs:
+  go_dependencies:
+    name: Go dependencies
+    runs-on: ubuntu-18.04
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Dump env
+      run: env | sort
+    - name: Dump GitHub context
+      env:
+        GITHUB_CONTEXT: ${{ toJson(github) }}
+      run: echo "$GITHUB_CONTEXT"
+    - name: Dump job context
+      env:
+        JOB_CONTEXT: ${{ toJson(job) }}
+      run: echo "$JOB_CONTEXT"
+    - name: Validate go deps
+      run: |
+        . bin/_tag.sh
+        for f in $( grep -lR --include=Dockerfile\* go-deps: . ) ; do
+          validate_go_deps_tag $f
+        done
+  go_lint:
+    name: Go lint
+    runs-on: ubuntu-18.04
+    container:
+      image: golang:1.13.4
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Go lint
+      env:
+        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
+      run: |
+        echo "$GITCOOKIE_SH" | bash
+        bin/lint --verbose
+  go_format:
+    name: Go format
+    runs-on: ubuntu-18.04
+    container:
+      image: golang:1.13.4
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Format
+      env:
+        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
+      run: |
+        echo "$GITCOOKIE_SH" | bash
+        bin/fmt
diff --git a/.github/workflows/workflow.yml b/.github/workflows/workflow.yml
deleted file mode 100644
index e595201b9..000000000
--- a/.github/workflows/workflow.yml
+++ /dev/null
@@ -1,533 +0,0 @@
-name: CI
-
-on:
-  pull_request: {}
-  push:
-    branches:
-    - master
-    tags:
-    - "*"
-
-# Jobs by event type and dependencies:
-
-# Unit tests for every master/tag push and PR
-#
-# validate_go_deps
-# go_unit_tests
-# go_lint
-# js_unit_tests
-
-# master/tag push and linkerd org PR: Docker build and integration tests
-#
-# docker_pull
-# docker_build
-# kind_setup
-#   -> kind_integration_host
-#     -> kind_cleanup
-
-# Forked repository PR: Docker build and integration tests
-#
-# docker_build
-#   -> kind_integration_github
-
-# Docker deploy and cloud integration tests for every master/tag push
-#
-#       -> docker_deploy
-#         -> cloud_integration
-
-jobs:
-
-  #
-  # Unit tests run for:
-  # - every master push
-  # - every tag push
-  # - every PR
-  #
-
-  validate_go_deps:
-    name: Validate go deps
-    runs-on: ubuntu-18.04
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    # for debugging
-    - name: Dump env
-      run: |
-        env | sort
-    - name: Dump GitHub context
-      env:
-        GITHUB_CONTEXT: ${{ toJson(github) }}
-      run: echo "$GITHUB_CONTEXT"
-    - name: Dump job context
-      env:
-        JOB_CONTEXT: ${{ toJson(job) }}
-      run: echo "$JOB_CONTEXT"
-    - name: Validate go deps
-      run: |
-        . bin/_tag.sh
-        for f in $( grep -lR --include=Dockerfile\* go-deps: . ) ; do
-          validate_go_deps_tag $f
-        done
-
-  go_unit_tests:
-    name: Go unit tests
-    runs-on: ubuntu-18.04
-    container:
-      image: golang:1.13.4
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Go unit tests
-      env:
-        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
-      run: |
-        echo "$GITCOOKIE_SH" | bash
-        # TODO: validate bin/protoc-go.sh does not dirty the repo
-        go test -cover -race -v -mod=readonly ./...
-
-  go_lint:
-    name: Go lint
-    runs-on: ubuntu-18.04
-    container:
-      image: golang:1.13.4
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Go lint
-      env:
-        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
-        # prevent OOM
-        GOGC: 20
-      run: |
-        echo "$GITCOOKIE_SH" | bash
-        bin/lint --verbose
-
-  go_fmt:
-    name: Go Format
-    runs-on: ubuntu-18.04
-    container:
-      image: golang:1.13.4
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Format
-      env:
-        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
-      run: |
-        echo "$GITCOOKIE_SH" | bash
-        bin/fmt
-
-  js_unit_tests:
-    name: JS unit tests
-    runs-on: ubuntu-18.04
-    container:
-      image: node:10.16.0-stretch
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Yarn setup
-      run: |
-        curl -o- -L https://yarnpkg.com/install.sh | bash -s -- --version 1.21.1 --network-concurrency 1
-    - name: JS unit tests
-      run: |
-        export PATH="$HOME/.yarn/bin:$PATH"
-        export NODE_ENV=test
-        bin/web
-        bin/web test --reporters=jest-dot-reporter
-
-  #
-  # Docker build and kind integration tests run for:
-  # - every master push
-  # - every tag push
-  # - every PR
-  #
-
-  docker_pull:
-    name: Docker pull
-    if: github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork
-    runs-on: ubuntu-18.04
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Docker SSH setup
-      run: |
-        mkdir -p ~/.ssh/
-        touch ~/.ssh/id && chmod 600 ~/.ssh/id
-        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
-        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
-        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
-        ssh linkerd-docker docker version
-    - name: Docker pull
-      env:
-        DOCKER_HOST: ssh://linkerd-docker
-      run: |
-        bin/docker pull gcr.io/linkerd-io/proxy-init:v1.3.1
-        bin/docker pull prom/prometheus:v2.11.1
-
-  docker_build:
-    name: Docker build
-    runs-on: ubuntu-18.04
-    env:
-      IMAGE_ARCHIVES_DIR: /home/runner/archives
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Docker SSH setup
-      if: github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork
-      run: |
-        mkdir -p ~/.ssh/
-        touch ~/.ssh/id && chmod 600 ~/.ssh/id
-        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
-        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
-        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
-        ssh linkerd-docker docker version
-        echo ::set-env name=DOCKER_HOST::ssh://linkerd-docker
-    - name: Build docker images
-      run: |
-        export PATH="`pwd`/bin:$PATH"
-        DOCKER_TRACE=1 bin/docker-build
-    - name: Create artifact with CLI and image archives
-      if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
-      run: |
-        mkdir -p $IMAGE_ARCHIVES_DIR
-        cp target/cli/linux/linkerd $IMAGE_ARCHIVES_DIR/
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        for img in controller grafana proxy web ; do
-          docker save "gcr.io/linkerd-io/$img:$TAG" > $IMAGE_ARCHIVES_DIR/$img.tar
-        done
-    # `with.path` values do not support environment variables yet, so an
-    # absolute path is used here.
-    #
-    # https://github.com/actions/upload-artifact/issues/8
-    - name: Upload artifact
-      if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
-      uses: actions/upload-artifact@v1
-      with:
-        name: image-archives
-        path: /home/runner/archives
-
-  kind_setup:
-    strategy:
-      max-parallel: 3
-      matrix:
-        integration_test: [deep, upgrade, helm, helm_upgrade, custom_domain, external_issuer]
-    name: Cluster setup (${{ matrix.integration_test }})
-    if: github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork
-    runs-on: ubuntu-18.04
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Docker SSH setup
-      run: |
-        mkdir -p ~/.ssh/
-        touch ~/.ssh/id && chmod 600 ~/.ssh/id
-        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
-        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
-        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
-        ssh linkerd-docker docker version
-    - name: Kind cluster setup
-      env:
-        DOCKER_HOST: ssh://linkerd-docker
-      run: |
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        export KIND_CLUSTER=github-$TAG-${{ matrix.integration_test }}
-        export KUBECONFIG=/tmp/kind-config-$KIND_CLUSTER
-        export CUSTOM_DOMAIN_CONFIG="test/testdata/custom_cluster_domain_config.yaml"
-        # retry cluster creation once in case of port conflict or kubeadm failure
-        if [ "${{ matrix.integration_test }}" == "custom_domain" ]
-        then
-          bin/kind create cluster --name=$KIND_CLUSTER --wait=2m --verbosity 3 --config=$CUSTOM_DOMAIN_CONFIG ||
-            bin/kind create cluster --name=$KIND_CLUSTER --wait=2m --verbosity 3 --config=$CUSTOM_DOMAIN_CONFIG
-        else
-          bin/kind create cluster --name=$KIND_CLUSTER --wait=2m --verbosity 3 ||
-            bin/kind create cluster --name=$KIND_CLUSTER --wait=2m --verbosity 3
-        fi
-
-  #
-  # Integration tests that run on the docker host are limited by the fact that
-  # while they all run in separate KinD clusters, they are on the same
-  # machine. The `strategy` context cannot be conditionally set, so there are
-  # separate `kind_integration_*` jobs. This allows the `kind_integration_github`
-  # job to run it's entire matrix in parallel.
-  #
-
-  kind_integration_host:
-    strategy:
-      max-parallel: 3
-      matrix:
-        integration_test: [deep, upgrade, helm, helm_upgrade, custom_domain, external_issuer]
-    needs: [docker_pull, docker_build, kind_setup]
-    name: Host int. tests (${{ matrix.integration_test }})
-    if: github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork
-    runs-on: ubuntu-18.04
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Docker SSH setup
-      run: |
-        mkdir -p ~/.ssh/
-        touch ~/.ssh/id && chmod 600 ~/.ssh/id
-        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
-        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
-        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
-        ssh linkerd-docker docker version
-    - name: Try to load cached Go modules
-      uses: actions/cache@v1
-      with:
-        path: ~/go/pkg/mod
-        key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
-        restore-keys: |
-          ${{ runner.os }}-go-
-    - name: Kind load docker images
-      run: |
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        export KIND_CLUSTER=github-$TAG-${{ matrix.integration_test }}
-        ssh -T linkerd-docker > /dev/null << EOF
-          # TODO: This is using the kind binary on the remote host.
-          # TODO: 'kind' still points to v0.5.1. When there are no more old CI branches depending on that
-          # we can replace it with v0.6.1. In the meantime we explicitly use 'kind-0.6.1'
-          kind-0.6.1 load docker-image gcr.io/linkerd-io/proxy-init:v1.3.1 --name=$KIND_CLUSTER
-          kind-0.6.1 load docker-image prom/prometheus:v2.11.1 --name=$KIND_CLUSTER
-          for IMG in controller grafana proxy web ; do
-            kind-0.6.1 load docker-image gcr.io/linkerd-io/\$IMG:$TAG --name=$KIND_CLUSTER
-          done
-        EOF
-    - name: Install linkerd CLI
-      env:
-        DOCKER_HOST: ssh://linkerd-docker
-      run: |
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        image="gcr.io/linkerd-io/cli-bin:$TAG"
-        id=$(bin/docker create $image)
-        bin/docker cp "$id:/out/linkerd-linux" "$HOME/.linkerd"
-        $HOME/.linkerd version --client
-        # validate CLI version matches the repo
-        [[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
-        echo "Installed Linkerd CLI version: $TAG"
-    - name: Run integration tests
-      env:
-        DOCKER_HOST: ssh://linkerd-docker
-        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
-      run: |
-        export PATH="`pwd`/bin:$PATH"
-        echo "$GITCOOKIE_SH" | bash
-        # TODO: pin Go version
-        go version
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        export KIND_CLUSTER=github-$TAG-${{ matrix.integration_test }}
-        # Restore kubeconfig from remote docker host.
-        mkdir -p $HOME/.kube
-        export KUBECONFIG=$HOME/.kube/kind-config-$KIND_CLUSTER
-        bin/kind export kubeconfig --name=$KIND_CLUSTER --kubeconfig $KUBECONFIG
-        # Start ssh tunnel to allow kubectl to connect via localhost.
-        export KIND_PORT=$(bin/kubectl config view -o jsonpath="{.clusters[?(@.name=='kind-$KIND_CLUSTER')].cluster.server}" | cut -d':' -f3)
-        echo "KIND_PORT: $KIND_PORT"
-        ssh -4 -N -L $KIND_PORT:localhost:$KIND_PORT linkerd-docker &
-        sleep 2 # Wait for ssh tunnel to come up.
-        bin/kubectl version --short # Test connection to kind cluster.
-        (
-          . bin/_test-run.sh
-          init_test_run $HOME/.linkerd
-          ${{ matrix.integration_test }}_integration_tests
-        )
-
-  kind_integration_github:
-    strategy:
-      matrix:
-        integration_test: [deep, upgrade, helm, helm_upgrade, custom_domain, external_issuer]
-    needs: [docker_build]
-    name: GitHub int. tests (${{ matrix.integration_test }})
-    if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
-    runs-on: ubuntu-18.04
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Download image archives
-      uses: actions/download-artifact@v1
-      with:
-        name: image-archives
-    - name: Try to load cached Go modules
-      uses: actions/cache@v1
-      with:
-        path: ~/go/pkg/mod
-        key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
-        restore-keys: |
-          ${{ runner.os }}-go-
-    - name: Setup KinD (default)
-      if: matrix.integration_test != 'custom_domain'
-      uses: engineerd/setup-kind@v0.3.0
-      with:
-        version: "v0.6.1"
-    - name: Setup KinD (custom_domain)
-      if: matrix.integration_test == 'custom_domain'
-      uses: engineerd/setup-kind@v0.3.0
-      with:
-        config: test/testdata/custom_cluster_domain_config.yaml
-        version: "v0.6.1"
-    - name: Load image archives into KinD
-      run: |
-        for img in controller grafana proxy web; do
-          kind load image-archive image-archives/$img.tar
-        done
-    - name: Install linkerd CLI
-      run: |
-        cp image-archives/linkerd $HOME/.linkerd
-        chmod +x $HOME/.linkerd
-    - name: Run integration tests
-      run: |
-        . bin/_test-run.sh
-        init_test_run $HOME/.linkerd
-        ${{ matrix.integration_test }}_integration_tests
-
-  kind_cleanup:
-    strategy:
-      fail-fast: false # always attempt to cleanup all clusters
-      matrix:
-        integration_test: [deep, upgrade, helm, helm_upgrade, custom_domain, external_issuer]
-    needs: [kind_integration_host]
-    name: Cluster cleanup (${{ matrix.integration_test }})
-    if: always() && (github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork)
-    runs-on: ubuntu-18.04
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    # for debugging
-    - name: Dump env
-      run: |
-        env | sort
-    - name: Dump GitHub context
-      env:
-        GITHUB_CONTEXT: ${{ toJson(github) }}
-      run: echo "$GITHUB_CONTEXT"
-    - name: Dump job context
-      env:
-        JOB_CONTEXT: ${{ toJson(job) }}
-      run: echo "$JOB_CONTEXT"
-    - name: Docker SSH setup
-      run: |
-        mkdir -p ~/.ssh/
-        touch ~/.ssh/id && chmod 600 ~/.ssh/id
-        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
-        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
-        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
-        ssh linkerd-docker docker version
-    - name: Kind cluster cleanup
-      env:
-        DOCKER_HOST: ssh://linkerd-docker
-      run: |
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        export KIND_CLUSTER=github-$TAG-${{ matrix.integration_test }}
-        bin/kind delete cluster --name=$KIND_CLUSTER
-
-  #
-  # Docker deploy and cloud integration tests run for:
-  # - every master push
-  # - every tag push
-  #
-
-  docker_deploy:
-    name: Docker deploy
-    if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags')
-    runs-on: ubuntu-18.04
-    needs: [validate_go_deps, go_unit_tests, go_lint, js_unit_tests, kind_integration_host, kind_cleanup]
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Configure gcloud
-      uses: linkerd/linkerd2-action-gcloud@v1.0.0
-      with:
-        cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
-        gcp_project: ${{ secrets.GCP_PROJECT }}
-        gcp_zone: ${{ secrets.GCP_ZONE }}
-    - name: Docker SSH setup
-      run: |
-        mkdir -p ~/.ssh/
-        touch ~/.ssh/id && chmod 600 ~/.ssh/id
-        echo "${{ secrets.DOCKER_SSH_CONFIG }}"  > ~/.ssh/config
-        echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
-        echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
-        ssh linkerd-docker docker version
-    - name: Docker push
-      env:
-        DOCKER_HOST: ssh://linkerd-docker
-      run: |
-        export PATH="`pwd`/bin:$PATH"
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        bin/docker-push-deps
-        bin/docker-push $TAG
-        bin/docker-retag-all $TAG master
-        bin/docker-push master
-
-  cloud_integration:
-    name: Cloud integration tests
-    if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags')
-    runs-on: ubuntu-18.04
-    needs: [docker_deploy]
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Install linkerd CLI
-      id: install_cli
-      run: |
-        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
-        image="gcr.io/linkerd-io/cli-bin:$TAG"
-        id=$(bin/docker create $image)
-        bin/docker cp "$id:/out/linkerd-linux" "$HOME/.linkerd"
-        $HOME/.linkerd version --client
-        # validate CLI version matches the repo
-        [[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
-        echo "Installed Linkerd CLI version: $TAG"
-        echo "::set-output name=tag::$TAG"
-    - name: Create GKE cluster
-      uses: linkerd/linkerd2-action-gcloud@v1.0.0
-      with:
-        cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
-        gcp_project: ${{ secrets.GCP_PROJECT }}
-        gcp_zone: ${{ secrets.GCP_ZONE }}
-        create: true
-        name: testing-${{ steps.install_cli.outputs.tag }}-${{ github.run_id }}
-    - name: Run integration tests
-      env:
-        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
-      run: |
-        export PATH="`pwd`/bin:$PATH"
-        echo "$GITCOOKIE_SH" | bash
-        version="$($HOME/.linkerd version --client --short | tr -cd '[:alnum:]-')"
-        bin/test-run $HOME/.linkerd linkerd-$version
-    - name: CNI tests
-      run: |
-        export TAG="$($HOME/.linkerd version --client --short)"
-        go test -cover -race -v -mod=readonly ./cni-plugin/test -integration-tests
-
-  # Helm chart artifact deploy run for:
-  # - every tag push
-  #
-
-  chart_deploy:
-    name: Helm chart deploy
-    if: startsWith(github.ref, 'refs/tags')
-    runs-on: ubuntu-18.04
-    needs: [cloud_integration]
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v2
-    - name: Configure gsutils
-      uses: linkerd/linkerd2-action-gcloud@v1.0.0
-      with:
-        cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
-        gcp_project: ${{ secrets.GCP_PROJECT }}
-        gcp_zone: ${{ secrets.GCP_ZONE }}
-    - name: Edge Helm chart creation and upload
-      if: startsWith(github.ref, 'refs/tags/edge')
-      run: |
-        mkdir -p target/helm
-        gsutil cp gs://helm.linkerd.io/edge/index.yaml target/helm/index-pre.yaml
-        bin/helm-build package
-        gsutil rsync target/helm gs://helm.linkerd.io/edge
-    - name: Stable Helm chart creation and upload
-      if: startsWith(github.ref, 'refs/tags/stable')
-      run: |
-        mkdir -p target/helm
-        gsutil cp gs://helm.linkerd.io/stable/index.yaml target/helm/index-pre.yaml
-        bin/helm-build package
-        gsutil rsync target/helm gs://helm.linkerd.io/stable
diff --git a/charts/linkerd2-cni/templates/cni-plugin.yaml b/charts/linkerd2-cni/templates/cni-plugin.yaml
index 003a39965..4521750d1 100644
--- a/charts/linkerd2-cni/templates/cni-plugin.yaml
+++ b/charts/linkerd2-cni/templates/cni-plugin.yaml
@@ -25,7 +25,7 @@ metadata:
   annotations:
     {{.Values.proxyInjectAnnotation}}: {{.Values.proxyInjectDisabled}}
   labels:
-    {{.Values.linkerdNamespaceLabel}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
     config.linkerd.io/admission-webhooks: disabled
 ---
 apiVersion: policy/v1beta1
@@ -33,7 +33,7 @@ kind: PodSecurityPolicy
 metadata:
   name: linkerd-{{.Values.namespace}}-cni
   labels:
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
 spec:
   allowPrivilegeEscalation: false
   fsGroup:
@@ -55,7 +55,7 @@ metadata:
   name: linkerd-cni
   namespace: {{.Values.namespace}}
   labels:
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: Role
@@ -63,7 +63,7 @@ metadata:
   name: linkerd-cni
   namespace: {{.Values.namespace}}
   labels:
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
 rules:
 - apiGroups: ['extensions', 'policy']
   resources: ['podsecuritypolicies']
@@ -77,7 +77,7 @@ metadata:
   name: linkerd-cni
   namespace: {{.Values.namespace}}
   labels:
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: Role
@@ -92,7 +92,7 @@ apiVersion: rbac.authorization.k8s.io/v1beta1
 metadata:
   name: linkerd-cni
   labels:
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
 rules:
 - apiGroups: [""]
   resources: ["pods", "nodes", "namespaces"]
@@ -103,7 +103,7 @@ kind: ClusterRoleBinding
 metadata:
   name: linkerd-cni
   labels:
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
@@ -119,7 +119,7 @@ metadata:
   name: linkerd-cni-config
   namespace: {{.Values.namespace}}
   labels:
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
 data:
   dest_cni_net_dir: "{{.Values.destCNINetDir}}"
   dest_cni_bin_dir: "{{.Values.destCNIBinDir}}"
@@ -161,7 +161,7 @@ metadata:
   namespace: {{.Values.namespace}}
   labels:
     k8s-app: linkerd-cni
-    {{.Values.cniResourceAnnotation}}: "true"
+    {{.Values.cniResourceLabel}}: "true"
   annotations:
     {{.Values.createdByAnnotation}}: {{.Values.cliVersion}}
 spec:
diff --git a/charts/linkerd2-cni/values.yaml b/charts/linkerd2-cni/values.yaml
index 3a3c502f9..adcc92b11 100644
--- a/charts/linkerd2-cni/values.yaml
+++ b/charts/linkerd2-cni/values.yaml
@@ -1,5 +1,5 @@
 namespace: linkerd-cni
-cniResourceAnnotation: linkerd.io/cni-resource
+cniResourceLabel: linkerd.io/cni-resource
 inboundProxyPort: 4143
 outboundProxyPort: 4140
 ignoreInboundPorts: ""
@@ -17,4 +17,3 @@ useWaitFlag:      false
 # namespace annotation and labels - do not edit
 proxyInjectAnnotation: linkerd.io/inject
 proxyInjectDisabled: disabled
-linkerdNamespaceLabel: linkerd.io/is-control-plane
diff --git a/cli/cmd/install-cni-plugin.go b/cli/cmd/install-cni-plugin.go
index 54eda64ae..f8edae2f7 100644
--- a/cli/cmd/install-cni-plugin.go
+++ b/cli/cmd/install-cni-plugin.go
@@ -76,8 +76,8 @@ func newCmdInstallCNIPlugin() *cobra.Command {
 
 	cmd := &cobra.Command{
 		Use:   "install-cni [flags]",
-		Short: "Output Kubernetes configs to install Linkerd CNI (experimental)",
-		Long: `Output Kubernetes configs to install Linkerd CNI (experimental).
+		Short: "Output Kubernetes configs to install Linkerd CNI",
+		Long: `Output Kubernetes configs to install Linkerd CNI.
 
 This command installs a DaemonSet into the Linkerd control plane. The DaemonSet
 copies the necessary linkerd-cni plugin binaries and configs onto the host. It
diff --git a/cli/cmd/install.go b/cli/cmd/install.go
index 721fafbef..ba351fd80 100644
--- a/cli/cmd/install.go
+++ b/cli/cmd/install.go
@@ -499,7 +499,7 @@ func (options *installOptions) allStageFlagSet() *pflag.FlagSet {
 	flags := pflag.NewFlagSet("all-stage", pflag.ExitOnError)
 
 	flags.BoolVar(&options.cniEnabled, "linkerd-cni-enabled", options.cniEnabled,
-		"Experimental: Omit the NET_ADMIN capability in the PSP and the proxy-init container when injecting the proxy; requires the linkerd-cni plugin to already be installed",
+		"Omit the NET_ADMIN capability in the PSP and the proxy-init container when injecting the proxy; requires the linkerd-cni plugin to already be installed",
 	)
 
 	flags.BoolVar(
diff --git a/cli/cmd/upgrade.go b/cli/cmd/upgrade.go
index 1ae754be7..8ad243f62 100644
--- a/cli/cmd/upgrade.go
+++ b/cli/cmd/upgrade.go
@@ -7,6 +7,7 @@ import (
 	"io/ioutil"
 	"os"
 	"strings"
+	"time"
 
 	"github.com/linkerd/linkerd2/pkg/config"
 	"github.com/linkerd/linkerd2/pkg/issuercerts"
@@ -26,11 +27,11 @@ import (
 )
 
 const (
-	okMessage              = "You're on your way to upgrading Linkerd!"
-	controlPlaneMessage    = "Don't forget to run `linkerd upgrade control-plane`!"
-	visitMessage           = "Visit this URL for further instructions: https://linkerd.io/upgrade/#nextsteps"
-	failMessage            = "For troubleshooting help, visit: https://linkerd.io/upgrade/#troubleshooting\n"
-	trustRootChangeMessage = "Rotating the trust anchors will affect existing proxies\nSee https://linkerd.io/2/tasks/rotating_identity_certificates/ for more information"
+	okMessage                = "You're on your way to upgrading Linkerd!"
+	controlPlaneMessage      = "Don't forget to run `linkerd upgrade control-plane`!"
+	visitMessage             = "Visit this URL for further instructions: https://linkerd.io/upgrade/#nextsteps"
+	failMessage              = "For troubleshooting help, visit: https://linkerd.io/upgrade/#troubleshooting\n"
+	trustAnchorChangeMessage = "Rotating the trust anchors will affect existing proxies\nSee https://linkerd.io/2/tasks/rotating_identity_certificates/ for more information"
 )
 
 type upgradeOptions struct {
@@ -66,7 +67,7 @@ func (options *upgradeOptions) upgradeOnlyFlagSet() *pflag.FlagSet {
 	)
 	flags.BoolVar(
 		&options.force, "force", options.force,
-		"Force upgrade operation even when issuer certificate does not work with the roots of all proxies",
+		"Force upgrade operation even when issuer certificate does not work with the trust anchors of all proxies",
 	)
 
 	return flags
@@ -196,7 +197,7 @@ func upgradeRunE(options *upgradeOptions, stage string, flags *pflag.FlagSet) er
 	buf.WriteTo(os.Stdout)
 
 	if options.identityOptions.trustPEMFile != "" {
-		fmt.Fprintf(os.Stderr, "\n%s %s\n", warnStatus, trustRootChangeMessage)
+		fmt.Fprintf(os.Stderr, "\n%s %s\n", warnStatus, trustAnchorChangeMessage)
 	}
 
 	fmt.Fprintf(os.Stderr, "\n%s %s\n", okStatus, okMessage)
@@ -396,10 +397,10 @@ func ensureIssuerCertWorksWithAllProxies(k kubernetes.Interface, cred *tls.Cred)
 		return err
 	}
 	for _, pod := range meshedPods {
-		roots, err := tls.DecodePEMCertPool(pod.Anchors)
+		anchors, err := tls.DecodePEMCertPool(pod.Anchors)
 
-		if roots != nil {
-			err = cred.Verify(roots, "")
+		if anchors != nil {
+			err = cred.Verify(anchors, "", time.Time{})
 		}
 
 		if err != nil {
@@ -408,7 +409,7 @@ func ensureIssuerCertWorksWithAllProxies(k kubernetes.Interface, cred *tls.Cred)
 	}
 
 	if len(problematicPods) > 0 {
-		errorMessageHeader := "You are attempting to use an issuer certificate which does not validate against the trust roots of the following pods:"
+		errorMessageHeader := "You are attempting to use an issuer certificate which does not validate against the trust anchors of the following pods:"
 		errorMessageFooter := "These pods do not have the current trust bundle and must be restarted.  Use the --force flag to proceed anyway (this will likely prevent those pods from sending or receiving traffic)."
 		return fmt.Errorf("%s\n\t%s\n%s", errorMessageHeader, strings.Join(problematicPods, "\n\t"), errorMessageFooter)
 	}
@@ -459,7 +460,7 @@ func (options *upgradeOptions) fetchIdentityValues(k kubernetes.Interface, idctx
 
 	cred, err := issuerData.VerifyAndBuildCreds("")
 	if err != nil {
-		return nil, fmt.Errorf("issuer certificate does not work with the provided roots: %s\nFor more information: https://linkerd.io/2/tasks/rotating_identity_certificates/", err)
+		return nil, fmt.Errorf("issuer certificate does not work with the provided anchors: %s\nFor more information: https://linkerd.io/2/tasks/rotating_identity_certificates/", err)
 	}
 	issuerData.Expiry = &cred.Crt.Certificate.NotAfter
 
@@ -545,8 +546,8 @@ func verifyWebhookTLS(value *charts.TLS, webhook string) error {
 	if err != nil {
 		return err
 	}
-	roots := crt.CertPool()
-	if err := crt.Verify(roots, webhookCommonName(webhook)); err != nil {
+	anchors := crt.CertPool()
+	if err := crt.Verify(anchors, webhookCommonName(webhook), time.Time{}); err != nil {
 		return err
 	}
 
diff --git a/controller/api/destination/watcher/endpoints_watcher.go b/controller/api/destination/watcher/endpoints_watcher.go
index 7e3c1da2d..875c99370 100644
--- a/controller/api/destination/watcher/endpoints_watcher.go
+++ b/controller/api/destination/watcher/endpoints_watcher.go
@@ -194,7 +194,20 @@ func (ew *EndpointsWatcher) addService(obj interface{}) {
 }
 
 func (ew *EndpointsWatcher) deleteService(obj interface{}) {
-	service := obj.(*corev1.Service)
+	service, ok := obj.(*corev1.Service)
+	if !ok {
+		tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
+		if !ok {
+			ew.log.Errorf("couldn't get object from DeletedFinalStateUnknown %#v", obj)
+			return
+		}
+		service, ok = tombstone.Obj.(*corev1.Service)
+		if !ok {
+			ew.log.Errorf("DeletedFinalStateUnknown contained object that is not a Service %#v", obj)
+			return
+		}
+	}
+
 	if service.Namespace == kubeSystem {
 		return
 	}
@@ -225,7 +238,20 @@ func (ew *EndpointsWatcher) addEndpoints(obj interface{}) {
 }
 
 func (ew *EndpointsWatcher) deleteEndpoints(obj interface{}) {
-	endpoints := obj.(*corev1.Endpoints)
+	endpoints, ok := obj.(*corev1.Endpoints)
+	if !ok {
+		tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
+		if !ok {
+			ew.log.Errorf("couldn't get object from DeletedFinalStateUnknown %#v", obj)
+			return
+		}
+		endpoints, ok = tombstone.Obj.(*corev1.Endpoints)
+		if !ok {
+			ew.log.Errorf("DeletedFinalStateUnknown contained object that is not an Endpoints %#v", obj)
+			return
+		}
+	}
+
 	if endpoints.Namespace == kubeSystem {
 		return
 	}
diff --git a/controller/api/destination/watcher/ip_watcher.go b/controller/api/destination/watcher/ip_watcher.go
index a2e0722e4..a38aa1330 100644
--- a/controller/api/destination/watcher/ip_watcher.go
+++ b/controller/api/destination/watcher/ip_watcher.go
@@ -116,7 +116,20 @@ func (iw *IPWatcher) addService(obj interface{}) {
 }
 
 func (iw *IPWatcher) deleteService(obj interface{}) {
-	service := obj.(*corev1.Service)
+	service, ok := obj.(*corev1.Service)
+	if !ok {
+		tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
+		if !ok {
+			iw.log.Errorf("couldn't get object from DeletedFinalStateUnknown %#v", obj)
+			return
+		}
+		service, ok = tombstone.Obj.(*corev1.Service)
+		if !ok {
+			iw.log.Errorf("DeletedFinalStateUnknown contained object that is not a Service %#v", obj)
+			return
+		}
+	}
+
 	if service.Namespace == kubeSystem {
 		return
 	}
@@ -141,7 +154,20 @@ func (iw *IPWatcher) addPod(obj interface{}) {
 }
 
 func (iw *IPWatcher) deletePod(obj interface{}) {
-	pod := obj.(*corev1.Pod)
+	pod, ok := obj.(*corev1.Pod)
+	if !ok {
+		tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
+		if !ok {
+			iw.log.Errorf("couldn't get object from DeletedFinalStateUnknown %#v", obj)
+			return
+		}
+		pod, ok = tombstone.Obj.(*corev1.Pod)
+		if !ok {
+			iw.log.Errorf("DeletedFinalStateUnknown contained object that is not a Pod %#v", obj)
+			return
+		}
+	}
+
 	if pod.Namespace == kubeSystem {
 		return
 	}
@@ -300,7 +326,6 @@ func (ss *serviceSubscriptions) updatePod(podSet PodSet) {
 func (ss *serviceSubscriptions) deletePod() {
 	ss.Lock()
 	defer ss.Unlock()
-
 	for listener, port := range ss.listeners {
 		listener.NoEndpoints(true) // Clear out previous endpoints.
 		listener.Add(singletonAddress(ss.clusterIP, port))
diff --git a/controller/api/destination/watcher/profile_watcher.go b/controller/api/destination/watcher/profile_watcher.go
index 6176431f5..4b4c33a9d 100644
--- a/controller/api/destination/watcher/profile_watcher.go
+++ b/controller/api/destination/watcher/profile_watcher.go
@@ -108,7 +108,20 @@ func (pw *ProfileWatcher) updateProfile(old interface{}, new interface{}) {
 }
 
 func (pw *ProfileWatcher) deleteProfile(obj interface{}) {
-	profile := obj.(*sp.ServiceProfile)
+	profile, ok := obj.(*sp.ServiceProfile)
+	if !ok {
+		tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
+		if !ok {
+			pw.log.Errorf("couldn't get object from DeletedFinalStateUnknown %#v", obj)
+			return
+		}
+		profile, ok = tombstone.Obj.(*sp.ServiceProfile)
+		if !ok {
+			pw.log.Errorf("DeletedFinalStateUnknown contained object that is not a ServiceProfile %#v", obj)
+			return
+		}
+	}
+
 	id := ProfileID{
 		Namespace: profile.Namespace,
 		Name:      profile.Name,
diff --git a/controller/api/destination/watcher/traffic_split_watcher.go b/controller/api/destination/watcher/traffic_split_watcher.go
index b563e6950..3b6c832d8 100644
--- a/controller/api/destination/watcher/traffic_split_watcher.go
+++ b/controller/api/destination/watcher/traffic_split_watcher.go
@@ -108,7 +108,20 @@ func (tsw *TrafficSplitWatcher) updateTrafficSplit(old interface{}, new interfac
 }
 
 func (tsw *TrafficSplitWatcher) deleteTrafficSplit(obj interface{}) {
-	split := obj.(*ts.TrafficSplit)
+	split, ok := obj.(*ts.TrafficSplit)
+	if !ok {
+		tombstone, ok := obj.(cache.DeletedFinalStateUnknown)
+		if !ok {
+			tsw.log.Errorf("couldn't get object from DeletedFinalStateUnknown %#v", obj)
+			return
+		}
+		split, ok = tombstone.Obj.(*ts.TrafficSplit)
+		if !ok {
+			tsw.log.Errorf("DeletedFinalStateUnknown contained object that is not a TrafficSplit %#v", obj)
+			return
+		}
+	}
+
 	id := ServiceID{
 		Name:      split.Spec.Service,
 		Namespace: split.Namespace,
diff --git a/controller/api/public/stat_summary.go b/controller/api/public/stat_summary.go
index fc2b1ffb5..c0400f049 100644
--- a/controller/api/public/stat_summary.go
+++ b/controller/api/public/stat_summary.go
@@ -453,17 +453,15 @@ func buildRequestLabels(req *pb.StatSummaryRequest) (labels model.LabelSet, labe
 }
 
 func buildTrafficSplitRequestLabels(req *pb.StatSummaryRequest) (labels model.LabelSet, labelNames model.LabelNames) {
-	// trafficsplit labels are always direction="outbound" with an optional namespace="value" if the -A flag is not used.
-	// if the --from or --to flags were used, we merge an additional ToResource or FromResource label.
-	// trafficsplit metrics results are always grouped by dst_service.
+	// Trafficsplit labels are always direction="outbound". If the --from or --to flags were used,
+	// we merge an additional ToResource or FromResource label. Trafficsplit metrics results are
+	// always grouped by dst_service.
+	// N.b. requests to a traffic split may come from any namespace so we do not do any filtering
+	// by namespace.
 	labels = model.LabelSet{
 		"direction": model.LabelValue("outbound"),
 	}
 
-	if req.Selector.Resource.Namespace != "" {
-		labels["namespace"] = model.LabelValue(req.Selector.Resource.Namespace)
-	}
-
 	switch out := req.Outbound.(type) {
 	case *pb.StatSummaryRequest_ToResource:
 		labels = labels.Merge(promDstQueryLabels(out.ToResource))
diff --git a/pkg/charts/cni/values.go b/pkg/charts/cni/values.go
index c4881ce94..27583e52b 100644
--- a/pkg/charts/cni/values.go
+++ b/pkg/charts/cni/values.go
@@ -16,7 +16,7 @@ const (
 // Values contains the top-level elements in the cni Helm chart
 type Values struct {
 	Namespace             string `json:"namespace"`
-	CNIResourceAnnotation string `json:"cniResourceAnnotation"`
+	CNIResourceLabel      string `json:"cniResourceLabel"`
 	InboundProxyPort      uint   `json:"inboundProxyPort"`
 	OutboundProxyPort     uint   `json:"outboundProxyPort"`
 	IgnoreInboundPorts    string `json:"ignoreInboundPorts"`
@@ -33,7 +33,6 @@ type Values struct {
 	UseWaitFlag           bool   `json:"useWaitFlag"`
 	ProxyInjectAnnotation string `json:"proxyInjectAnnotation"`
 	ProxyInjectDisabled   string `json:"proxyInjectDisabled"`
-	LinkerdNamespaceLabel string `json:"linkerdNamespaceLabel"`
 }
 
 // NewValues returns a new instance of the Values type.
diff --git a/pkg/healthcheck/healthcheck.go b/pkg/healthcheck/healthcheck.go
index 27cdd4d38..d23906714 100644
--- a/pkg/healthcheck/healthcheck.go
+++ b/pkg/healthcheck/healthcheck.go
@@ -343,7 +343,7 @@ type HealthChecker struct {
 	linkerdConfig    *configPb.All
 	uuid             string
 	issuerCert       *tls.Cred
-	roots            []*x509.Certificate
+	trustAnchors     []*x509.Certificate
 	cniDaemonSet     *appsv1.DaemonSet
 }
 
@@ -879,58 +879,58 @@ func (hc *HealthChecker) allCategories() []category {
 					hintAnchor:  "l5d-identity-cert-config-valid",
 					fatal:       true,
 					check: func(context.Context) (err error) {
-						hc.issuerCert, hc.roots, err = hc.checkCertificatesConfig()
+						hc.issuerCert, hc.trustAnchors, err = hc.checkCertificatesConfig()
 						return
 					},
 				},
 				{
-					description: "trust roots are using supported crypto algorithm",
-					hintAnchor:  "l5d-identity-roots-use-supported-crypto",
+					description: "trust anchors are using supported crypto algorithm",
+					hintAnchor:  "l5d-identity-trustAnchors-use-supported-crypto",
 					fatal:       true,
 					check: func(context.Context) error {
-						var invalidRoots []string
-						for _, root := range hc.roots {
-							if err := issuercerts.CheckCertAlgoRequirements(root); err != nil {
-								invalidRoots = append(invalidRoots, fmt.Sprintf("* %v %s %s", root.SerialNumber, root.Subject.CommonName, err))
+						var invalidAnchors []string
+						for _, anchor := range hc.trustAnchors {
+							if err := issuercerts.CheckCertAlgoRequirements(anchor); err != nil {
+								invalidAnchors = append(invalidAnchors, fmt.Sprintf("* %v %s %s", anchor.SerialNumber, anchor.Subject.CommonName, err))
 							}
 						}
-						if len(invalidRoots) > 0 {
-							return fmt.Errorf("Invalid roots:\n\t%s", strings.Join(invalidRoots, "\n\t"))
+						if len(invalidAnchors) > 0 {
+							return fmt.Errorf("Invalid trustAnchors:\n\t%s", strings.Join(invalidAnchors, "\n\t"))
 						}
 						return nil
 					},
 				},
 				{
-					description: "trust roots are within their validity period",
-					hintAnchor:  "l5d-identity-roots-are-time-valid",
+					description: "trust anchors are within their validity period",
+					hintAnchor:  "l5d-identity-trustAnchors-are-time-valid",
 					fatal:       true,
 					check: func(ctx context.Context) error {
-						var expiredRoots []string
-						for _, root := range hc.roots {
-							if err := issuercerts.CheckCertValidityPeriod(root); err != nil {
-								expiredRoots = append(expiredRoots, fmt.Sprintf("* %v %s %s", root.SerialNumber, root.Subject.CommonName, err))
+						var expiredAnchors []string
+						for _, anchor := range hc.trustAnchors {
+							if err := issuercerts.CheckCertValidityPeriod(anchor); err != nil {
+								expiredAnchors = append(expiredAnchors, fmt.Sprintf("* %v %s %s", anchor.SerialNumber, anchor.Subject.CommonName, err))
 							}
 						}
-						if len(expiredRoots) > 0 {
-							return fmt.Errorf("Invalid roots:\n\t%s", strings.Join(expiredRoots, "\n\t"))
+						if len(expiredAnchors) > 0 {
+							return fmt.Errorf("Invalid anchors:\n\t%s", strings.Join(expiredAnchors, "\n\t"))
 						}
 
 						return nil
 					},
 				},
 				{
-					description: "trust roots are valid for at least 60 days",
-					hintAnchor:  "l5d-identity-roots-not-expiring-soon",
+					description: "trust anchors are valid for at least 60 days",
+					hintAnchor:  "l5d-identity-trustAnchors-not-expiring-soon",
 					warning:     true,
 					check: func(ctx context.Context) error {
-						var expiringRoots []string
-						for _, root := range hc.roots {
-							if err := issuercerts.CheckExpiringSoon(root); err != nil {
-								expiringRoots = append(expiringRoots, fmt.Sprintf("* %v %s %s", root.SerialNumber, root.Subject.CommonName, err))
+						var expiringAnchors []string
+						for _, anchor := range hc.trustAnchors {
+							if err := issuercerts.CheckExpiringSoon(anchor); err != nil {
+								expiringAnchors = append(expiringAnchors, fmt.Sprintf("* %v %s %s", anchor.SerialNumber, anchor.Subject.CommonName, err))
 							}
 						}
-						if len(expiringRoots) > 0 {
-							return fmt.Errorf("Roots expiring soon:\n\t%s", strings.Join(expiringRoots, "\n\t"))
+						if len(expiringAnchors) > 0 {
+							return fmt.Errorf("Anchors expiring soon:\n\t%s", strings.Join(expiringAnchors, "\n\t"))
 						}
 						return nil
 					},
@@ -969,10 +969,10 @@ func (hc *HealthChecker) allCategories() []category {
 					},
 				},
 				{
-					description: "issuer cert is issued by the trust root",
-					hintAnchor:  "l5d-identity-issuer-cert-issued-by-trust-root",
+					description: "issuer cert is issued by the trust anchor",
+					hintAnchor:  "l5d-identity-issuer-cert-issued-by-trust-anchor",
 					check: func(ctx context.Context) error {
-						return hc.issuerCert.Verify(tls.CertificatesToPool(hc.roots), hc.issuerIdentity())
+						return hc.issuerCert.Verify(tls.CertificatesToPool(hc.trustAnchors), hc.issuerIdentity(), time.Time{})
 					},
 				},
 			},
@@ -1400,7 +1400,7 @@ func (hc *HealthChecker) checkCertificatesConfig() (*tls.Cred, []*x509.Certifica
 		data, err = issuercerts.FetchIssuerData(hc.kubeAPI, idctx.TrustAnchorsPem, hc.ControlPlaneNamespace)
 	} else {
 		data, err = issuercerts.FetchExternalIssuerData(hc.kubeAPI, hc.ControlPlaneNamespace)
-		// ensure trust trustRoots in config matches whats in the secret
+		// ensure trust anchors in config matches whats in the secret
 		if data != nil && strings.TrimSpace(idctx.TrustAnchorsPem) != strings.TrimSpace(data.TrustAnchors) {
 			errFormat := "IdentityContext.TrustAnchorsPem does not match %s in %s"
 			err = fmt.Errorf(errFormat, k8s.IdentityIssuerTrustAnchorsNameExternal, k8s.IdentityIssuerSecretName)
@@ -1416,12 +1416,12 @@ func (hc *HealthChecker) checkCertificatesConfig() (*tls.Cred, []*x509.Certifica
 		return nil, nil, err
 	}
 
-	roots, err := tls.DecodePEMCertificates(data.TrustAnchors)
+	anchors, err := tls.DecodePEMCertificates(data.TrustAnchors)
 	if err != nil {
 		return nil, nil, err
 	}
 
-	return issuerCreds, roots, nil
+	return issuerCreds, anchors, nil
 }
 
 // FetchLinkerdConfigMap retrieves the `linkerd-config` ConfigMap from
@@ -1617,7 +1617,7 @@ func (hc *HealthChecker) checkPodSecurityPolicies(shouldExist bool) error {
 	return checkResources("PodSecurityPolicies", objects, []string{fmt.Sprintf("linkerd-%s-control-plane", hc.ControlPlaneNamespace)}, shouldExist)
 }
 
-// MeshedPodIdentityData contains meshed pod details + root anchors of the proxy
+// MeshedPodIdentityData contains meshed pod details + trust anchors of the proxy
 type MeshedPodIdentityData struct {
 	Name      string
 	Namespace string
diff --git a/pkg/identity/service.go b/pkg/identity/service.go
index 1914832e3..208eefefa 100644
--- a/pkg/identity/service.go
+++ b/pkg/identity/service.go
@@ -29,6 +29,7 @@ const (
 	EnvTrustAnchors  = "LINKERD2_PROXY_IDENTITY_TRUST_ANCHORS"
 	eventTypeSkipped = "IssuerUpdateSkipped"
 	eventTypeUpdated = "IssuerUpdated"
+	eventTypeFailed  = "IssuerValidationFailed"
 )
 
 type (
@@ -113,7 +114,7 @@ func (svc *Service) loadCredentials() (tls.Issuer, error) {
 		return nil, fmt.Errorf("failed to read CA from disk: %s", err)
 	}
 
-	if err := creds.Crt.Verify(svc.trustAnchors, svc.expectedName); err != nil {
+	if err := creds.Crt.Verify(svc.trustAnchors, svc.expectedName, time.Time{}); err != nil {
 		return nil, fmt.Errorf("failed to verify issuer credentials for '%s' with trust anchors: %s", svc.expectedName, err)
 	}
 
@@ -148,7 +149,7 @@ func (svc *Service) ensureIssuerStillValid() error {
 	issuer := *svc.issuer
 	switch is := issuer.(type) {
 	case *tls.CA:
-		return is.Cred.Verify(svc.trustAnchors, svc.expectedName)
+		return is.Cred.Verify(svc.trustAnchors, svc.expectedName, time.Time{})
 	default:
 		return fmt.Errorf("unsupported issuer type. Expected *tls.CA, got %v", is)
 	}
@@ -171,7 +172,9 @@ func (svc *Service) Certify(ctx context.Context, req *pb.CertifyRequest) (*pb.Ce
 	}
 
 	if err := svc.ensureIssuerStillValid(); err != nil {
-		log.Errorf("could not process CSR because of CA cert validation failure: %s", err)
+		log.Errorf("could not process CSR because of CA cert validation failure: %s - CSR Identity : %s", err, reqIdentity)
+		message := fmt.Sprintf("%s - CSR Identity : %s", err.Error(), reqIdentity)
+		svc.recordEvent(v1.EventTypeWarning, eventTypeFailed, message)
 		return nil, err
 	}
 
diff --git a/pkg/issuercerts/issuercerts.go b/pkg/issuercerts/issuercerts.go
index ad1b853f6..c5ded8d3a 100644
--- a/pkg/issuercerts/issuercerts.go
+++ b/pkg/issuercerts/issuercerts.go
@@ -18,7 +18,7 @@ import (
 const keyMissingError = "key %s containing the %s needs to exist in secret %s if --identity-external-issuer=%v"
 const expirationWarningThresholdInDays = 60
 
-// IssuerCertData holds the root cert data used by the CA
+// IssuerCertData holds the trust anchors cert data used by the CA
 type IssuerCertData struct {
 	TrustAnchors string
 	IssuerCrt    string
@@ -164,12 +164,12 @@ func (ic *IssuerCertData) VerifyAndBuildCreds(dnsName string) (*tls.Cred, error)
 		return nil, fmt.Errorf("issuer cert is not a CA")
 	}
 
-	roots, err := tls.DecodePEMCertPool(ic.TrustAnchors)
+	anchors, err := tls.DecodePEMCertPool(ic.TrustAnchors)
 	if err != nil {
 		return nil, err
 	}
 
-	if err := creds.Verify(roots, dnsName); err != nil {
+	if err := creds.Verify(anchors, dnsName, time.Time{}); err != nil {
 		return nil, err
 	}
 
diff --git a/pkg/tls/cred.go b/pkg/tls/cred.go
index d46218a6c..d88da77fc 100644
--- a/pkg/tls/cred.go
+++ b/pkg/tls/cred.go
@@ -10,6 +10,7 @@ import (
 	"errors"
 	"fmt"
 	"io/ioutil"
+	"time"
 )
 
 type (
@@ -83,13 +84,21 @@ func (crt *Crt) CertPool() *x509.CertPool {
 }
 
 // Verify the validity of the provided certificate
-func (crt *Crt) Verify(roots *x509.CertPool, name string) error {
+func (crt *Crt) Verify(roots *x509.CertPool, name string, currentTime time.Time) error {
 	i := x509.NewCertPool()
 	for _, c := range crt.TrustChain {
 		i.AddCert(c)
 	}
-	vo := x509.VerifyOptions{Roots: roots, Intermediates: i, DNSName: name}
+	vo := x509.VerifyOptions{Roots: roots, Intermediates: i, DNSName: name, CurrentTime: currentTime}
 	_, err := crt.Certificate.Verify(vo)
+
+	if currentTime.IsZero() {
+		currentTime = time.Now()
+	}
+
+	if crtExpiryError(err) {
+		return fmt.Errorf("%s - Current Time : %s - Invalid before %s - Invalid After %s", err, currentTime, crt.Certificate.NotBefore, crt.Certificate.NotAfter)
+	}
 	return err
 }
 
@@ -221,3 +230,12 @@ func DecodePEMCrt(txt string) (*Crt, error) {
 
 	return &crt, nil
 }
+
+func crtExpiryError(err error) bool {
+	switch v := err.(type) {
+	case x509.CertificateInvalidError:
+		return v.Reason == x509.Expired
+	default:
+		return false
+	}
+}
diff --git a/web/app/.eslintrc b/web/app/.eslintrc
index 852a1cabd..82d6db977 100644
--- a/web/app/.eslintrc
+++ b/web/app/.eslintrc
@@ -83,7 +83,6 @@
     "react/jsx-props-no-spreading": 0,
 
     // These rules represent best practices that we're not adhering to, and should be enabled
-    "jsx-a11y/click-events-have-key-events": 0,
-    "no-param-reassign": 0
+    "no-param-reassign": ["error", { "props": false }]
   }
 }
diff --git a/web/app/js/components/Popover.jsx b/web/app/js/components/Popover.jsx
index 763d3c4b0..50f815149 100644
--- a/web/app/js/components/Popover.jsx
+++ b/web/app/js/components/Popover.jsx
@@ -21,6 +21,13 @@ class ClickablePopover extends React.Component {
     this.setState({ anchorEl: event.currentTarget });
   };
 
+  handleKeyPress = event => {
+    if (event.key === 'Enter') {
+      this.setState({ anchorEl: event.currentTarget });
+    }
+  }
+
+
   handleClose = () => {
     this.setState({ anchorEl: null });
   };
@@ -36,6 +43,7 @@ class ClickablePopover extends React.Component {
           aria-owns={open ? 'clickable-popover' : null}
           aria-haspopup="true"
           onClick={this.handleClick}
+          onKeyPress={this.handleKeyPress}
           role="button"
           tabIndex={0}>
           {baseContent}
diff --git a/web/app/js/components/util/ApiHelpers.jsx b/web/app/js/components/util/ApiHelpers.jsx
index 0ccea34e6..99a27dd0a 100644
--- a/web/app/js/components/util/ApiHelpers.jsx
+++ b/web/app/js/components/util/ApiHelpers.jsx
@@ -70,29 +70,32 @@ const ApiHelpers = (pathPrefix, defaultMetricsWindow = '1m') => {
 
   // for getting json api results
   const apiFetch = path => {
+    let path_ = path;
     if (!_isEmpty(pathPrefix)) {
-      path = `${pathPrefix}${path}`;
+      path_ = `${pathPrefix}${path_}`;
     }
 
-    return makeCancelable(fetch(path), r => r.json());
+    return makeCancelable(fetch(path_), r => r.json());
   };
 
   // for getting yaml api results
   const apiFetchYAML = path => {
+    let path_ = path;
     if (!_isEmpty(pathPrefix)) {
-      path = `${pathPrefix}${path}`;
+      path_ = `${pathPrefix}${path_}`;
     }
 
-    return makeCancelable(fetch(path), r => r.text());
+    return makeCancelable(fetch(path_), r => r.text());
   };
 
   // for getting non-json results
   const prefixedUrl = path => {
+    let path_ = path;
     if (!_isEmpty(pathPrefix)) {
-      path = `${pathPrefix}${path}`;
+      path_ = `${pathPrefix}${path_}`;
     }
 
-    return path;
+    return path_;
   };
 
   const getMetricsWindow = () => metricsWindow;
@@ -104,14 +107,15 @@ const ApiHelpers = (pathPrefix, defaultMetricsWindow = '1m') => {
   };
 
   const fetchMetrics = path => {
-    if (path.indexOf('window') === -1) {
-      if (path.indexOf('?') === -1) {
-        path = `${path}?window=${getMetricsWindow()}`;
+    let path_ = path;
+    if (path_.indexOf('window') === -1) {
+      if (path_.indexOf('?') === -1) {
+        path_ = `${path_}?window=${getMetricsWindow()}`;
       } else {
-        path = `${path}&window=${getMetricsWindow()}`;
+        path_ = `${path_}&window=${getMetricsWindow()}`;
       }
     }
-    return apiFetch(path);
+    return apiFetch(path_);
   };
 
   const fetchPods = namespace => {
diff --git a/web/app/js/components/util/TapUtils.jsx b/web/app/js/components/util/TapUtils.jsx
index bb3eee6cd..26d0e5676 100644
--- a/web/app/js/components/util/TapUtils.jsx
+++ b/web/app/js/components/util/TapUtils.jsx
@@ -159,12 +159,13 @@ const tapEventKey = (d, eventType) => {
   produce octets given an ip address
 */
 const decodeIPToOctets = ip => {
-  ip = parseInt(ip, 10);
+  const ip_ = parseInt(ip, 10);
+
   return [
-    (ip >> 24) & 255,
-    (ip >> 16) & 255,
-    (ip >> 8) & 255,
-    ip & 255,
+    (ip_ >> 24) & 255,
+    (ip_ >> 16) & 255,
+    (ip_ >> 8) & 255,
+    ip_ & 255,
   ];
 };
 
diff --git a/web/app/js/components/util/Utils.js b/web/app/js/components/util/Utils.js
index 2b6d882a0..4a3c21295 100644
--- a/web/app/js/components/util/Utils.js
+++ b/web/app/js/components/util/Utils.js
@@ -54,8 +54,9 @@ export const formatLatencyMs = m => {
 * Add commas to a number (converting it to a string in the process)
 */
 export function addCommas(nStr) {
-  nStr += '';
-  const x = nStr.split('.');
+  let nStr_ = nStr;
+  nStr_ += '';
+  const x = nStr_.split('.');
   let x1 = x[0];
   const x2 = x.length > 1 ? `.${x[1]}` : '';
   const rgx = /(\d+)(\d{3})/;
@@ -76,25 +77,26 @@ export const roundNumber = (num, dec) => {
 * Shorten and style number
 */
 export const styleNum = (number, unit = '', truncate = true) => {
-  if (Number.isNaN(number)) {
+  let number_ = number;
+  if (Number.isNaN(number_)) {
     return 'N/A';
   }
 
-  if (truncate && number > 999999999) {
-    number = roundNumber(number / 1000000000.0, 3);
-    return `${addCommas(number)}G${unit}`;
-  } else if (truncate && number > 999999) {
-    number = roundNumber(number / 1000000.0, 3);
-    return `${addCommas(number)}M${unit}`;
-  } else if (truncate && number > 999) {
-    number = roundNumber(number / 1000.0, 3);
-    return `${addCommas(number)}k${unit}`;
-  } else if (number > 999) {
-    number = roundNumber(number, 0);
-    return addCommas(number) + unit;
+  if (truncate && number_ > 999999999) {
+    number_ = roundNumber(number_ / 1000000000.0, 3);
+    return `${addCommas(number_)}G${unit}`;
+  } else if (truncate && number_ > 999999) {
+    number_ = roundNumber(number_ / 1000000.0, 3);
+    return `${addCommas(number_)}M${unit}`;
+  } else if (truncate && number_ > 999) {
+    number_ = roundNumber(number_ / 1000.0, 3);
+    return `${addCommas(number_)}k${unit}`;
+  } else if (number_ > 999) {
+    number_ = roundNumber(number_, 0);
+    return addCommas(number_) + unit;
   } else {
-    number = roundNumber(number, 2);
-    return addCommas(number) + unit;
+    number_ = roundNumber(number_, 2);
+    return addCommas(number_) + unit;
   }
 };
 
@@ -120,9 +122,9 @@ export const toClassName = name => {
 */
 export const regexFilterString = input => {
   // make input lower case and strip out unwanted characters
-  input = input.replace(/[^A-Z0-9/.\-_*]/gi, '').toLowerCase();
+  const input_ = input.replace(/[^A-Z0-9/.\-_*]/gi, '').toLowerCase();
   // replace "*" in input with wildcard
-  return new RegExp(input.replace(/[*]/g, '.+'));
+  return new RegExp(input_.replace(/[*]/g, '.+'));
 };
 
 /*
@@ -218,12 +220,12 @@ export const isResource = name => {
   produce octets given an ip address
 */
 const decodeIPToOctets = ip => {
-  ip = parseInt(ip, 10);
+  const ip_ = parseInt(ip, 10);
   return [
-    (ip >> 24) & 255,
-    (ip >> 16) & 255,
-    (ip >> 8) & 255,
-    ip & 255,
+    (ip_ >> 24) & 255,
+    (ip_ >> 16) & 255,
+    (ip_ >> 8) & 255,
+    ip_ & 255,
   ];
 };
 


Test patch:
diff --git a/.github/workflows/unit_tests.yml b/.github/workflows/unit_tests.yml
new file mode 100644
index 000000000..cc273c448
--- /dev/null
+++ b/.github/workflows/unit_tests.yml
@@ -0,0 +1,40 @@
+name: Unit tests
+on:
+  pull_request: {}
+  push:
+    branches:
+    - master
+    tags:
+    - "*"
+jobs:
+  go_unit_tests:
+    name: Go unit tests
+    runs-on: ubuntu-18.04
+    container:
+      image: golang:1.13.4
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Go unit tests
+      env:
+        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
+      run: |
+        echo "$GITCOOKIE_SH" | bash
+        # TODO: validate bin/protoc-go.sh does not dirty the repo
+        go test -cover -race -v -mod=readonly ./...
+  js_unit_tests:
+    name: JS unit tests
+    runs-on: ubuntu-18.04
+    container:
+      image: node:10.16.0-stretch
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v2
+    - name: Yarn setup
+      run: curl -o- -L https://yarnpkg.com/install.sh | bash -s -- --version 1.21.1 --network-concurrency 1
+    - name: JS unit tests
+      run: |
+        export PATH="$HOME/.yarn/bin:$PATH"
+        export NODE_ENV=test
+        bin/web
+        bin/web test --reporters=jest-dot-reporter
diff --git a/bin/test-cleanup b/bin/test-cleanup
index 622b67302..4e84ddb4e 100755
--- a/bin/test-cleanup
+++ b/bin/test-cleanup
@@ -14,6 +14,12 @@ if [ -z "${namespaces_helm=$(kubectl --context=$k8s_context get ns -oname -l lin
   echo 'no helm namespaces found' >&2
 fi
 
+echo "cleaning up CNI namespaces in k8s-context [${k8s_context}]"
+
+if [ -z "${namespaces_cni=$(kubectl --context=$k8s_context get ns -oname -l linkerd.io/cni-resource)}" ]; then
+  echo 'no CNI namespaces found' >&2
+fi
+
 echo "cleaning up data-plane namespaces in k8s-context [${k8s_context}]"
 
 if [ -z "${namespaces_dataplane=$(kubectl --context=$k8s_context get ns -oname -l linkerd.io/is-test-data-plane)}" ]; then
@@ -52,8 +58,8 @@ if [ -z "${apiservices=$(kubectl --context=$k8s_context get apiservices -l linke
   echo 'no apiservices found' >&2
 fi
 
-if [[ $namespaces_controlplane || $namespaces_helm || $namespaces_dataplane || $clusterrolebindings || $clusterrolebindings_helm || $clusterroles || $webhookconfigs || $validatingconfigs || $podsecuritypolicies || $customresourcedefinitions || $apiservices ]]; then
-  kubectl --context=$k8s_context delete $namespaces_controlplane $namespaces_helm $namespaces_dataplane $clusterrolebindings $clusterrolebindings_helm $clusterroles $webhookconfigs $validatingconfigs $podsecuritypolicies $customresourcedefinitions $apiservices
+if [[ $namespaces_controlplane || $namespaces_cni || $namespaces_helm || $namespaces_dataplane || $clusterrolebindings || $clusterrolebindings_helm || $clusterroles || $webhookconfigs || $validatingconfigs || $podsecuritypolicies || $customresourcedefinitions || $apiservices ]]; then
+  kubectl --context=$k8s_context delete $namespaces_controlplane $namespaces_cni $namespaces_helm $namespaces_dataplane $clusterrolebindings $clusterrolebindings_helm $clusterroles $webhookconfigs $validatingconfigs $podsecuritypolicies $customresourcedefinitions $apiservices
 fi
 
 echo "cleaning up rolebindings in kube-system namespace in k8s-context [${k8s_context}]"
diff --git a/cli/cmd/install_cni_helm_test.go b/cli/cmd/install_cni_helm_test.go
index 98f9129a3..10a55c97b 100644
--- a/cli/cmd/install_cni_helm_test.go
+++ b/cli/cmd/install_cni_helm_test.go
@@ -34,8 +34,7 @@ func testRenderCniHelm(t *testing.T, chart *pb.Chart, goldenFileName string) {
 	overrideJSON :=
 		`{
 			"namespace": "linkerd-test",
-  			"controllerNamespaceLabel": "linkerd.io/control-plane-ns-test",
-  			"cniResourceAnnotation": "linkerd.io/cni-resource-test",
+  			"cniResourceLabel": "linkerd.io/cni-resource-test",
   			"inboundProxyPort": 1234,
   			"outboundProxyPort": 5678,
 			"createdByAnnotation": "linkerd.io/created-by-test",
diff --git a/cli/cmd/testdata/install-cni-plugin_default.golden b/cli/cmd/testdata/install-cni-plugin_default.golden
index 10237b8be..9985df0d6 100644
--- a/cli/cmd/testdata/install-cni-plugin_default.golden
+++ b/cli/cmd/testdata/install-cni-plugin_default.golden
@@ -5,7 +5,7 @@ metadata:
   annotations:
     linkerd.io/inject: disabled
   labels:
-    linkerd.io/is-control-plane: "true"
+    linkerd.io/cni-resource: "true"
     config.linkerd.io/admission-webhooks: disabled
 ---
 apiVersion: policy/v1beta1
diff --git a/cli/cmd/testdata/install-cni-plugin_fully_configured.golden b/cli/cmd/testdata/install-cni-plugin_fully_configured.golden
index e2c177477..2f9314d9a 100644
--- a/cli/cmd/testdata/install-cni-plugin_fully_configured.golden
+++ b/cli/cmd/testdata/install-cni-plugin_fully_configured.golden
@@ -5,7 +5,7 @@ metadata:
   annotations:
     linkerd.io/inject: disabled
   labels:
-    linkerd.io/is-control-plane: "true"
+    linkerd.io/cni-resource: "true"
     config.linkerd.io/admission-webhooks: disabled
 ---
 apiVersion: policy/v1beta1
diff --git a/cli/cmd/testdata/install-cni-plugin_fully_configured_equal_dsts.golden b/cli/cmd/testdata/install-cni-plugin_fully_configured_equal_dsts.golden
index 996fe69a3..8a9a422e3 100644
--- a/cli/cmd/testdata/install-cni-plugin_fully_configured_equal_dsts.golden
+++ b/cli/cmd/testdata/install-cni-plugin_fully_configured_equal_dsts.golden
@@ -5,7 +5,7 @@ metadata:
   annotations:
     linkerd.io/inject: disabled
   labels:
-    linkerd.io/is-control-plane: "true"
+    linkerd.io/cni-resource: "true"
     config.linkerd.io/admission-webhooks: disabled
 ---
 apiVersion: policy/v1beta1
diff --git a/cli/cmd/testdata/install_cni_helm_output.golden b/cli/cmd/testdata/install_cni_helm_output.golden
index 4a3a08274..4b178da9b 100644
--- a/cli/cmd/testdata/install_cni_helm_output.golden
+++ b/cli/cmd/testdata/install_cni_helm_output.golden
@@ -7,7 +7,7 @@ metadata:
   annotations:
     linkerd.io/inject: disabled
   labels:
-    linkerd.io/is-control-plane: "true"
+    linkerd.io/cni-resource-test: "true"
     config.linkerd.io/admission-webhooks: disabled
 ---
 apiVersion: policy/v1beta1
diff --git a/cli/cmd/upgrade_test.go b/cli/cmd/upgrade_test.go
index 5e52be12f..b43a5deca 100644
--- a/cli/cmd/upgrade_test.go
+++ b/cli/cmd/upgrade_test.go
@@ -1015,7 +1015,7 @@ spec:
 			},
 			"upgrade_overwrite_issuer.golden",
 			// use Errorf here to stop the linter complaining that our error message is capitalized...
-			fmt.Errorf("%s", "You are attempting to use an issuer certificate which does not validate against the trust roots of the following pods:\n\t* some-namespace/backend-wrong-anchors\nThese pods do not have the current trust bundle and must be restarted.  Use the --force flag to proceed anyway (this will likely prevent those pods from sending or receiving traffic)."),
+			fmt.Errorf("%s", "You are attempting to use an issuer certificate which does not validate against the trust anchors of the following pods:\n\t* some-namespace/backend-wrong-anchors\nThese pods do not have the current trust bundle and must be restarted.  Use the --force flag to proceed anyway (this will likely prevent those pods from sending or receiving traffic)."),
 			func(options *upgradeOptions) {
 				options.identityOptions.crtPEMFile = filepath.Join("testdata", "valid-crt.pem")
 				options.identityOptions.keyPEMFile = filepath.Join("testdata", "valid-key.pem")
diff --git a/controller/api/destination/watcher/endpoints_watcher_test.go b/controller/api/destination/watcher/endpoints_watcher_test.go
index eb3f9942f..6bd966e95 100644
--- a/controller/api/destination/watcher/endpoints_watcher_test.go
+++ b/controller/api/destination/watcher/endpoints_watcher_test.go
@@ -6,6 +6,8 @@ import (
 	"testing"
 
 	"github.com/linkerd/linkerd2/controller/k8s"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
 	logging "github.com/sirupsen/logrus"
 )
@@ -500,3 +502,118 @@ status:
 		})
 	}
 }
+
+func TestEndpointsWatcherDeletion(t *testing.T) {
+	k8sConfigs := []string{`
+apiVersion: v1
+kind: Service
+metadata:
+  name: name1
+  namespace: ns
+spec:
+  type: LoadBalancer
+  ports:
+  - port: 8989`,
+		`
+apiVersion: v1
+kind: Endpoints
+metadata:
+  name: name1
+  namespace: ns
+subsets:
+- addresses:
+  - ip: 172.17.0.12
+    targetRef:
+      kind: Pod
+      name: name1-1
+      namespace: ns
+  ports:
+  - port: 8989`,
+		`
+apiVersion: v1
+kind: Pod
+metadata:
+  name: name1-1
+  namespace: ns
+status:
+  phase: Running
+  podIP: 172.17.0.12`}
+
+	for _, tt := range []struct {
+		serviceType      string
+		k8sConfigs       []string
+		id               ServiceID
+		hostname         string
+		port             Port
+		objectToDelete   interface{}
+		deletingServices bool
+	}{
+		{
+			serviceType:    "can delete endpoints",
+			k8sConfigs:     k8sConfigs,
+			id:             ServiceID{Name: "name1", Namespace: "ns"},
+			port:           8989,
+			hostname:       "name1-1",
+			objectToDelete: &corev1.Endpoints{ObjectMeta: metav1.ObjectMeta{Name: "name1", Namespace: "ns"}},
+		},
+		{
+			serviceType:    "can delete endpoints when wrapped in a DeletedFinalStateUnknown",
+			k8sConfigs:     k8sConfigs,
+			id:             ServiceID{Name: "name1", Namespace: "ns"},
+			port:           8989,
+			hostname:       "name1-1",
+			objectToDelete: &corev1.Endpoints{ObjectMeta: metav1.ObjectMeta{Name: "name1", Namespace: "ns"}},
+		},
+		{
+			serviceType:      "can delete services",
+			k8sConfigs:       k8sConfigs,
+			id:               ServiceID{Name: "name1", Namespace: "ns"},
+			port:             8989,
+			hostname:         "name1-1",
+			objectToDelete:   &corev1.Service{ObjectMeta: metav1.ObjectMeta{Name: "name1", Namespace: "ns"}},
+			deletingServices: true,
+		},
+		{
+			serviceType:      "can delete services when wrapped in a DeletedFinalStateUnknown",
+			k8sConfigs:       k8sConfigs,
+			id:               ServiceID{Name: "name1", Namespace: "ns"},
+			port:             8989,
+			hostname:         "name1-1",
+			objectToDelete:   &corev1.Service{ObjectMeta: metav1.ObjectMeta{Name: "name1", Namespace: "ns"}},
+			deletingServices: true,
+		},
+	} {
+
+		tt := tt // pin
+		t.Run("subscribes listener to "+tt.serviceType, func(t *testing.T) {
+			k8sAPI, err := k8s.NewFakeAPI(tt.k8sConfigs...)
+			if err != nil {
+				t.Fatalf("NewFakeAPI returned an error: %s", err)
+			}
+
+			watcher := NewEndpointsWatcher(k8sAPI, logging.WithField("test", t.Name()))
+
+			k8sAPI.Sync()
+
+			listener := newBufferingEndpointListener()
+
+			err = watcher.Subscribe(tt.id, tt.port, tt.hostname, listener)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			if tt.deletingServices {
+				watcher.deleteService(tt.objectToDelete)
+
+			} else {
+				watcher.deleteEndpoints(tt.objectToDelete)
+			}
+
+			if !listener.noEndpointsCalled {
+				t.Fatal("Expected NoEndpoints to be Called")
+			}
+		})
+
+	}
+
+}
diff --git a/controller/api/destination/watcher/ip_watcher_test.go b/controller/api/destination/watcher/ip_watcher_test.go
index 5bdc61416..43f9b420e 100644
--- a/controller/api/destination/watcher/ip_watcher_test.go
+++ b/controller/api/destination/watcher/ip_watcher_test.go
@@ -4,9 +4,13 @@ import (
 	"sort"
 	"testing"
 
+	"k8s.io/client-go/tools/cache"
+
 	"github.com/linkerd/linkerd2/controller/k8s"
 
 	logging "github.com/sirupsen/logrus"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
 func TestIPWatcher(t *testing.T) {
@@ -460,3 +464,132 @@ status:
 		})
 	}
 }
+
+func TestIPWatcherDeletion(t *testing.T) {
+
+	podK8sConfig := []string{
+		`
+apiVersion: v1
+kind: Pod
+metadata:
+  name: name1-1
+  namespace: ns
+  ownerReferences:
+  - kind: ReplicaSet
+    name: rs-1
+status:
+  phase: Running
+  podIP: 172.17.0.12`,
+	}
+
+	serviceK8sConfig := []string{
+		`
+apiVersion: v1
+kind: Service
+metadata:
+  name: name1
+  namespace: ns
+spec:
+  type: LoadBalancer
+  clusterIP: 192.168.210.92
+  ports:
+  - port: 8989`,
+		`
+apiVersion: v1
+kind: Endpoints
+metadata:
+  name: name1
+  namespace: ns
+subsets:
+- addresses:
+  - ip: 172.17.0.12
+    targetRef:
+      kind: Pod
+      name: name1-1
+      namespace: ns
+`,
+		`
+apiVersion: v1
+kind: Pod
+metadata:
+  name: name1-1
+  namespace: ns
+  ownerReferences:
+  - kind: ReplicaSet
+    name: rs-1
+status:
+  phase: Running
+  podIP: 172.17.0.12`,
+	}
+
+	for _, tt := range []struct {
+		description    string
+		k8sConfigs     []string
+		host           string
+		port           Port
+		objectToDelete interface{}
+		deletingPod    bool
+	}{
+		{
+			description:    "can delete pods",
+			k8sConfigs:     podK8sConfig,
+			host:           "172.17.0.12",
+			port:           8989,
+			objectToDelete: &corev1.Pod{ObjectMeta: metav1.ObjectMeta{Name: "name1-1", Namespace: "ns"}, Status: corev1.PodStatus{PodIP: "172.17.0.12"}},
+			deletingPod:    true,
+		},
+		{
+			description:    "can delete pods wrapped in a DeletedFinalStateUnknown",
+			k8sConfigs:     podK8sConfig,
+			host:           "172.17.0.12",
+			port:           8989,
+			objectToDelete: cache.DeletedFinalStateUnknown{Obj: &corev1.Pod{ObjectMeta: metav1.ObjectMeta{Name: "name1-1", Namespace: "ns"}, Status: corev1.PodStatus{PodIP: "172.17.0.12"}}},
+			deletingPod:    true,
+		},
+		{
+			description:    "can delete services",
+			k8sConfigs:     serviceK8sConfig,
+			host:           "192.168.210.92",
+			port:           8989,
+			objectToDelete: &corev1.Service{ObjectMeta: metav1.ObjectMeta{Name: "name1", Namespace: "ns"}, Spec: corev1.ServiceSpec{ClusterIP: "192.168.210.92"}},
+		},
+		{
+			description:    "can delete services wrapped in a DeletedFinalStateUnknown",
+			k8sConfigs:     serviceK8sConfig,
+			host:           "192.168.210.92",
+			port:           8989,
+			objectToDelete: cache.DeletedFinalStateUnknown{Obj: &corev1.Service{ObjectMeta: metav1.ObjectMeta{Name: "name1", Namespace: "ns"}, Spec: corev1.ServiceSpec{ClusterIP: "192.168.210.92"}}},
+		},
+	} {
+
+		tt := tt // pin
+		t.Run("subscribes listener to "+tt.description, func(t *testing.T) {
+			k8sAPI, err := k8s.NewFakeAPI(tt.k8sConfigs...)
+			if err != nil {
+				t.Fatalf("NewFakeAPI returned an error: %s", err)
+			}
+
+			endpoints := NewEndpointsWatcher(k8sAPI, logging.WithField("test", t.Name()))
+			watcher := NewIPWatcher(k8sAPI, endpoints, logging.WithField("test", t.Name()))
+
+			k8sAPI.Sync()
+
+			listener := newBufferingEndpointListener()
+
+			err = watcher.Subscribe(tt.host, tt.port, listener)
+			if err != nil {
+				t.Fatal(err)
+			}
+
+			if tt.deletingPod {
+				watcher.deletePod(tt.objectToDelete)
+			} else {
+				watcher.deleteService(tt.objectToDelete)
+			}
+
+			if !listener.noEndpointsCalled {
+				t.Fatal("Expected NoEndpoints to be Called")
+			}
+		})
+	}
+}
diff --git a/controller/api/destination/watcher/profile_watcher_test.go b/controller/api/destination/watcher/profile_watcher_test.go
index c54cc1db9..e908ce6ac 100644
--- a/controller/api/destination/watcher/profile_watcher_test.go
+++ b/controller/api/destination/watcher/profile_watcher_test.go
@@ -3,21 +3,41 @@ package watcher
 import (
 	"testing"
 
+	"k8s.io/client-go/tools/cache"
+
 	sp "github.com/linkerd/linkerd2/controller/gen/apis/serviceprofile/v1alpha2"
 	"github.com/linkerd/linkerd2/controller/k8s"
 	logging "github.com/sirupsen/logrus"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
-func TestProfileWatcher(t *testing.T) {
-	for _, tt := range []struct {
-		name             string
-		k8sConfigs       []string
-		id               ProfileID
-		expectedProfiles []*sp.ServiceProfileSpec
-	}{
-		{
-			name: "service profile",
-			k8sConfigs: []string{`
+var testServiceProfile = sp.ServiceProfile{
+	ObjectMeta: metav1.ObjectMeta{
+		Name:      "foobar.ns.svc.cluster.local",
+		Namespace: "linkerd",
+	},
+	Spec: sp.ServiceProfileSpec{
+		Routes: []*sp.RouteSpec{
+			{
+				Condition: &sp.RequestMatch{
+					PathRegex: "/x/y/z",
+				},
+				ResponseClasses: []*sp.ResponseClass{
+					{
+						Condition: &sp.ResponseMatch{
+							Status: &sp.Range{
+								Min: 500,
+							},
+						},
+						IsFailure: true,
+					},
+				},
+			},
+		},
+	},
+}
+
+var testServiceProfileResource = `
 apiVersion: linkerd.io/v1alpha2
 kind: ServiceProfile
 metadata:
@@ -31,29 +51,21 @@ spec:
     - condition:
         status:
           min: 500
-      isFailure: true`,
-			},
-			id: ProfileID{Name: "foobar.ns.svc.cluster.local", Namespace: "linkerd"},
+      isFailure: true`
+
+func TestProfileWatcherUpdates(t *testing.T) {
+	for _, tt := range []struct {
+		name             string
+		k8sConfigs       []string
+		id               ProfileID
+		expectedProfiles []*sp.ServiceProfileSpec
+	}{
+		{
+			name:       "service profile",
+			k8sConfigs: []string{testServiceProfileResource},
+			id:         ProfileID{Name: testServiceProfile.Name, Namespace: testServiceProfile.Namespace},
 			expectedProfiles: []*sp.ServiceProfileSpec{
-				{
-					Routes: []*sp.RouteSpec{
-						{
-							Condition: &sp.RequestMatch{
-								PathRegex: "/x/y/z",
-							},
-							ResponseClasses: []*sp.ResponseClass{
-								{
-									Condition: &sp.ResponseMatch{
-										Status: &sp.Range{
-											Min: 500,
-										},
-									},
-									IsFailure: true,
-								},
-							},
-						},
-					},
-				},
+				&testServiceProfile.Spec,
 			},
 		},
 		{
@@ -94,3 +106,46 @@ spec:
 		})
 	}
 }
+
+func TestProfileWatcherDeletes(t *testing.T) {
+	for _, tt := range []struct {
+		name           string
+		k8sConfigs     []string
+		id             ProfileID
+		objectToDelete interface{}
+	}{
+		{
+			name:           "can delete service profiles",
+			k8sConfigs:     []string{testServiceProfileResource},
+			id:             ProfileID{Name: testServiceProfile.Name, Namespace: testServiceProfile.Namespace},
+			objectToDelete: &testServiceProfile,
+		},
+		{
+			name:           "can delete service profiles wrapped in a DeletedFinalStateUnknown",
+			k8sConfigs:     []string{testServiceProfileResource},
+			id:             ProfileID{Name: testServiceProfile.Name, Namespace: testServiceProfile.Namespace},
+			objectToDelete: cache.DeletedFinalStateUnknown{Obj: &testServiceProfile},
+		},
+	} {
+		tt := tt // pin
+		t.Run(tt.name, func(t *testing.T) {
+			k8sAPI, err := k8s.NewFakeAPI(tt.k8sConfigs...)
+			if err != nil {
+				t.Fatalf("NewFakeAPI returned an error: %s", err)
+			}
+
+			watcher := NewProfileWatcher(k8sAPI, logging.WithField("test", t.Name()))
+			k8sAPI.Sync()
+
+			listener := NewDeletingProfileListener()
+
+			watcher.Subscribe(tt.id, listener)
+
+			watcher.deleteProfile(tt.objectToDelete)
+
+			if listener.NumDeletes != 1 {
+				t.Fatalf("Expected to get 1 deletes but got %v", listener.NumDeletes)
+			}
+		})
+	}
+}
diff --git a/controller/api/destination/watcher/test_util.go b/controller/api/destination/watcher/test_util.go
index ca8f825d4..1b31ca266 100644
--- a/controller/api/destination/watcher/test_util.go
+++ b/controller/api/destination/watcher/test_util.go
@@ -8,6 +8,27 @@ import (
 	sp "github.com/linkerd/linkerd2/controller/gen/apis/serviceprofile/v1alpha2"
 )
 
+// DeletingProfileListener implements ProfileUpdateListener and registers
+// deletions. Useful for unit testing
+type DeletingProfileListener struct {
+	NumDeletes int
+}
+
+// NewDeletingProfileListener creates a new NewDeletingProfileListener.
+func NewDeletingProfileListener() *DeletingProfileListener {
+	return &DeletingProfileListener{
+		NumDeletes: 0,
+	}
+}
+
+// Update registers a deletion
+func (dpl *DeletingProfileListener) Update(profile *sp.ServiceProfile) {
+	if profile == nil {
+		dpl.NumDeletes = dpl.NumDeletes + 1
+
+	}
+}
+
 // BufferingProfileListener implements ProfileUpdateListener and stores updates
 // in a slice.  Useful for unit tests.
 type BufferingProfileListener struct {
diff --git a/controller/api/destination/watcher/traffic_split_watcher_test.go b/controller/api/destination/watcher/traffic_split_watcher_test.go
index 4c1eed9c7..8b2a4c741 100644
--- a/controller/api/destination/watcher/traffic_split_watcher_test.go
+++ b/controller/api/destination/watcher/traffic_split_watcher_test.go
@@ -3,10 +3,13 @@ package watcher
 import (
 	"testing"
 
+	"k8s.io/client-go/tools/cache"
+
 	ts "github.com/deislabs/smi-sdk-go/pkg/apis/split/v1alpha1"
 	"github.com/linkerd/linkerd2/controller/k8s"
 	logging "github.com/sirupsen/logrus"
 	"k8s.io/apimachinery/pkg/api/resource"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
 type bufferingTrafficSplitListener struct {
@@ -23,16 +26,25 @@ func (btsl *bufferingTrafficSplitListener) UpdateTrafficSplit(split *ts.TrafficS
 	btsl.splits = append(btsl.splits, split)
 }
 
-func TestTrafficSplitWatcher(t *testing.T) {
-	for _, tt := range []struct {
-		name           string
-		k8sConfigs     []string
-		service        ServiceID
-		expectedSplits []*ts.TrafficSplitSpec
-	}{
-		{
-			name: "traffic split",
-			k8sConfigs: []string{`
+type deletingTrafficSplitListener struct {
+	NumDeletes int
+}
+
+func newDeletingTrafficSplitListener() *deletingTrafficSplitListener {
+	return &deletingTrafficSplitListener{
+		NumDeletes: 0,
+	}
+}
+
+func (dpl *deletingTrafficSplitListener) UpdateTrafficSplit(ts *ts.TrafficSplit) {
+	if ts == nil {
+		dpl.NumDeletes = dpl.NumDeletes + 1
+
+	}
+}
+
+var (
+	testTrafficSplitResource = `
 apiVersion: split.smi-spec.io/v1alpha1
 kind: TrafficSplit
 metadata:
@@ -44,27 +56,44 @@ spec:
   - service: foo-v1
     weight: 500m
   - service: foo-v2
-    weight: 500m`,
+    weight: 500m`
+
+	testTrafficSplit = ts.TrafficSplit{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "split",
+			Namespace: "ns",
+		},
+		Spec: ts.TrafficSplitSpec{
+			Service: "foo",
+			Backends: []ts.TrafficSplitBackend{
+				{
+					Service: "foo-v1",
+					Weight:  resource.MustParse("500m"),
+				},
+				{
+					Service: "foo-v2",
+					Weight:  resource.MustParse("500m"),
+				},
 			},
+		},
+	}
+)
+
+func TestTrafficSplitWatcher(t *testing.T) {
+	for _, tt := range []struct {
+		name           string
+		k8sConfigs     []string
+		service        ServiceID
+		expectedSplits []*ts.TrafficSplitSpec
+	}{
+		{
+			name:       "traffic split",
+			k8sConfigs: []string{testTrafficSplitResource},
 			service: ServiceID{
 				Name:      "foo",
 				Namespace: "ns",
 			},
-			expectedSplits: []*ts.TrafficSplitSpec{
-				{
-					Service: "foo",
-					Backends: []ts.TrafficSplitBackend{
-						{
-							Service: "foo-v1",
-							Weight:  resource.MustParse("500m"),
-						},
-						{
-							Service: "foo-v2",
-							Weight:  resource.MustParse("500m"),
-						},
-					},
-				},
-			},
+			expectedSplits: []*ts.TrafficSplitSpec{&testTrafficSplit.Spec},
 		},
 		{
 			name:       "no traffic split",
@@ -107,3 +136,52 @@ spec:
 		})
 	}
 }
+
+func TestTrafficSplitWatcherDelete(t *testing.T) {
+	for _, tt := range []struct {
+		name           string
+		k8sConfigs     []string
+		service        ServiceID
+		objectToDelete interface{}
+	}{
+		{
+			name:       "can delete a traffic splits",
+			k8sConfigs: []string{testTrafficSplitResource},
+			service: ServiceID{
+				Name:      "foo",
+				Namespace: "ns",
+			},
+			objectToDelete: &testTrafficSplit,
+		},
+		{
+			name:       "can delete a traffic splits wrapped in a DeletedFinalStateUnknown",
+			k8sConfigs: []string{testTrafficSplitResource},
+			service: ServiceID{
+				Name:      "foo",
+				Namespace: "ns",
+			},
+			objectToDelete: cache.DeletedFinalStateUnknown{Obj: &testTrafficSplit},
+		},
+	} {
+		tt := tt // pin
+		t.Run(tt.name, func(t *testing.T) {
+			k8sAPI, err := k8s.NewFakeAPI(tt.k8sConfigs...)
+			if err != nil {
+				t.Fatalf("NewFakeAPI returned an error: %s", err)
+			}
+
+			watcher := NewTrafficSplitWatcher(k8sAPI, logging.WithField("test", t.Name()))
+
+			k8sAPI.Sync()
+
+			listener := newDeletingTrafficSplitListener()
+
+			watcher.Subscribe(tt.service, listener)
+
+			watcher.deleteTrafficSplit(tt.objectToDelete)
+			if listener.NumDeletes != 1 {
+				t.Fatalf("Expected to get 1 deletes but got %v", listener.NumDeletes)
+			}
+		})
+	}
+}
diff --git a/pkg/healthcheck/healthcheck_test.go b/pkg/healthcheck/healthcheck_test.go
index 0ce06776e..777ea8faa 100644
--- a/pkg/healthcheck/healthcheck_test.go
+++ b/pkg/healthcheck/healthcheck_test.go
@@ -2475,7 +2475,7 @@ func runIdentityCheckTestCase(t *testing.T, testID int, testDescription string,
 		_, hc.linkerdConfig, _ = hc.checkLinkerdConfigConfigMap()
 
 		if testDescription != "certificate config is valid" {
-			hc.issuerCert, hc.roots, _ = hc.checkCertificatesConfig()
+			hc.issuerCert, hc.trustAnchors, _ = hc.checkCertificatesConfig()
 		}
 
 		if err != nil {
@@ -2607,22 +2607,22 @@ func TestLinkerdIdentityCheckCertValidity(t *testing.T) {
 		expectedOutput   []string
 	}{
 		{
-			checkerToTest:    "trust roots are within their validity period",
-			checkDescription: "fails when the only root cert is not valid yet",
+			checkerToTest:    "trust anchors are within their validity period",
+			checkDescription: "fails when the only anchor is not valid yet",
 			lifespan: &lifeSpan{
 				starts: time.Date(2100, 1, 1, 1, 1, 1, 1, time.UTC),
 				ends:   time.Date(2101, 1, 1, 1, 1, 1, 1, time.UTC),
 			},
-			expectedOutput: []string{"linkerd-identity-test-cat trust roots are within their validity period: Invalid roots:\n\t* 1 identity.linkerd.cluster.local not valid before: 2100-01-01T01:00:51Z"},
+			expectedOutput: []string{"linkerd-identity-test-cat trust anchors are within their validity period: Invalid anchors:\n\t* 1 identity.linkerd.cluster.local not valid before: 2100-01-01T01:00:51Z"},
 		},
 		{
-			checkerToTest:    "trust roots are within their validity period",
-			checkDescription: "fails when the only root cert is expired",
+			checkerToTest:    "trust anchors are within their validity period",
+			checkDescription: "fails when the only trust anchor is expired",
 			lifespan: &lifeSpan{
 				starts: time.Date(1989, 1, 1, 1, 1, 1, 1, time.UTC),
 				ends:   time.Date(1990, 1, 1, 1, 1, 1, 1, time.UTC),
 			},
-			expectedOutput: []string{"linkerd-identity-test-cat trust roots are within their validity period: Invalid roots:\n\t* 1 identity.linkerd.cluster.local not valid anymore. Expired on 1990-01-01T01:01:11Z"},
+			expectedOutput: []string{"linkerd-identity-test-cat trust anchors are within their validity period: Invalid anchors:\n\t* 1 identity.linkerd.cluster.local not valid anymore. Expired on 1990-01-01T01:01:11Z"},
 		},
 		{
 			checkerToTest:    "issuer cert is within its validity period",
@@ -2654,11 +2654,11 @@ func TestLinkerdIdentityCheckCertValidity(t *testing.T) {
 }
 
 func TestLinkerdIdentityCheckWrongDns(t *testing.T) {
-	expectedOutput := []string{"linkerd-identity-test-cat issuer cert is issued by the trust root: x509: certificate is valid for wrong.linkerd.cluster.local, not identity.linkerd.cluster.local"}
+	expectedOutput := []string{"linkerd-identity-test-cat issuer cert is issued by the trust anchor: x509: certificate is valid for wrong.linkerd.cluster.local, not identity.linkerd.cluster.local"}
 	issuerData := createIssuerData("wrong.linkerd.cluster.local", time.Now().AddDate(-1, 0, 0), time.Now().AddDate(1, 0, 0))
 	fakeConfigMap := getFakeConfigMap(k8s.IdentityIssuerSchemeLinkerd, issuerData)
 	fakeSecret := getFakeSecret(k8s.IdentityIssuerSchemeLinkerd, issuerData)
-	runIdentityCheckTestCase(t, 0, "fails when cert dns is wrong", "issuer cert is issued by the trust root", fakeConfigMap, fakeSecret, expectedOutput)
+	runIdentityCheckTestCase(t, 0, "fails when cert dns is wrong", "issuer cert is issued by the trust anchor", fakeConfigMap, fakeSecret, expectedOutput)
 
 }
 
diff --git a/pkg/tls/cred_test.go b/pkg/tls/cred_test.go
index 576bf942c..a888cc6c0 100644
--- a/pkg/tls/cred_test.go
+++ b/pkg/tls/cred_test.go
@@ -1,7 +1,10 @@
 package tls
 
 import (
+	"crypto/x509"
+	"fmt"
 	"testing"
+	"time"
 )
 
 func newRoot(t *testing.T) CA {
@@ -26,7 +29,7 @@ func TestCrtRoundtrip(t *testing.T) {
 		t.Fatalf("Failed to decode PEM Crt: %s", err)
 	}
 
-	if err := crt.Verify(rootTrust, "endentity.test"); err != nil {
+	if err := crt.Verify(rootTrust, "endentity.test", time.Time{}); err != nil {
 		t.Fatal("Failed to verify round-tripped certificate")
 	}
 }
@@ -47,3 +50,71 @@ func TestCredEncodeCertificateAndTrustChain(t *testing.T) {
 		t.Errorf("Encoded Certificate And TrustChain does not match expected output")
 	}
 }
+
+func TestCrtExpiry(t *testing.T) {
+	root := newRoot(t)
+	rootTrust := root.Cred.Crt.CertPool()
+
+	cred, err := root.GenerateEndEntityCred("expired.test")
+	if err != nil {
+		t.Fatalf("failed to create end entity cred: %s", err)
+	}
+
+	crt, err := DecodePEMCrt(cred.Crt.EncodePEM())
+	if err != nil {
+		t.Fatalf("Failed to decode PEM Crt: %s", err)
+	}
+
+	fakeExpiryError := x509.CertificateInvalidError{Reason: x509.Expired}
+
+	//need to remove seconds and nanoseconds for testing returned error
+	now := time.Now()
+
+	testCases := []struct {
+		currentTime time.Time
+		notBefore   time.Time
+		notAfter    time.Time
+		expected    string
+	}{
+		//cert not valid yet
+		{
+			currentTime: now,
+			notAfter:    now.AddDate(0, 0, 20),
+			notBefore:   now.AddDate(0, 0, 10),
+			expected:    fmt.Sprintf("%s - Current Time : %s - Invalid before %s - Invalid After %s", fakeExpiryError.Error(), now, now.AddDate(0, 0, 10), now.AddDate(0, 0, 20)),
+		},
+		//cert has expired
+		{
+			currentTime: now,
+			notAfter:    now.AddDate(0, 0, -10),
+			notBefore:   now.AddDate(0, 0, -20),
+			expected:    fmt.Sprintf("%s - Current Time : %s - Invalid before %s - Invalid After %s", fakeExpiryError.Error(), now, now.AddDate(0, 0, -20), now.AddDate(0, 0, -10)),
+		},
+		// cert is valid
+		{
+			currentTime: time.Time{},
+			notAfter:    crt.Certificate.NotAfter,
+			notBefore:   crt.Certificate.NotBefore,
+			expected:    "",
+		},
+	}
+
+	for i, tc := range testCases {
+		tc := tc //pin
+		t.Run(fmt.Sprintf("%d", i), func(t *testing.T) {
+			//explicitly kill the certificate
+			crt.Certificate.NotBefore = tc.notBefore
+			crt.Certificate.NotAfter = tc.notAfter
+
+			if err := crt.Verify(rootTrust, "expired.test", tc.currentTime); err != nil {
+				if err.Error() != tc.expected {
+					t.Logf("Returned error : %s\n", err)
+					t.Logf("Expected error : %s\n", tc.expected)
+					t.Fatal("test case failed")
+				}
+			} else {
+				t.Log("no error on verification")
+			}
+		})
+	}
+}
diff --git a/test/install_test.go b/test/install_test.go
index 2dcfdee30..a4fac2037 100644
--- a/test/install_test.go
+++ b/test/install_test.go
@@ -92,7 +92,7 @@ var (
 		`(Liveness|Readiness) probe failed: Get http://.*: dial tcp .*: connect: connection refused`,
 		`(Liveness|Readiness) probe failed: Get http://.*: read tcp .*: read: connection reset by peer`,
 		`(Liveness|Readiness) probe failed: Get http://.*: net/http: request canceled .*\(Client\.Timeout exceeded while awaiting headers\)`,
-		`Failed to update endpoint .*-upgrade/linkerd-.*: Operation cannot be fulfilled on endpoints "linkerd-.*": the object has been modified; please apply your changes to the latest version and try again`,
+		`Failed to update endpoint .*/linkerd-.*: Operation cannot be fulfilled on endpoints "linkerd-.*": the object has been modified; please apply your changes to the latest version and try again`,
 		`error killing pod: failed to "KillPodSandbox" for ".*" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to destroy network for sandbox \\".*\\": could not teardown (ipv4|ipv6) dnat: running \[/usr/sbin/(iptables|ip6tables) -t nat -X CNI-DN-.* --wait\]: exit status 1: (iptables|ip6tables): No chain/target/match by that name\.\\n"`,
 	}, "|"))
 
@@ -754,9 +754,7 @@ func TestEvents(t *testing.T) {
 		}
 
 		evtStr := fmt.Sprintf("Reason: [%s] Object: [%s] Message: [%s]", e.Reason, e.InvolvedObject.Name, e.Message)
-		if knownEventWarningsRegex.MatchString(e.Message) {
-			t.Logf("Found known warning event: %s", evtStr)
-		} else {
+		if !knownEventWarningsRegex.MatchString(e.Message) {
 			unknownEvents = append(unknownEvents, evtStr)
 		}
 	}
diff --git a/test/testdata/check.golden b/test/testdata/check.golden
index c22570ce1..18861f3b9 100644
--- a/test/testdata/check.golden
+++ b/test/testdata/check.golden
@@ -32,13 +32,13 @@ linkerd-config
 linkerd-identity
 ----------------
 √ certificate config is valid
-√ trust roots are using supported crypto algorithm
-√ trust roots are within their validity period
-√ trust roots are valid for at least 60 days
+√ trust anchors are using supported crypto algorithm
+√ trust anchors are within their validity period
+√ trust anchors are valid for at least 60 days
 √ issuer cert is using supported crypto algorithm
 √ issuer cert is within its validity period
 √ issuer cert is valid for at least 60 days
-√ issuer cert is issued by the trust root
+√ issuer cert is issued by the trust anchor
 
 linkerd-api
 -----------
diff --git a/test/testdata/check.proxy.golden b/test/testdata/check.proxy.golden
index 2e6a15469..368aaf55b 100644
--- a/test/testdata/check.proxy.golden
+++ b/test/testdata/check.proxy.golden
@@ -32,13 +32,13 @@ linkerd-config
 linkerd-identity
 ----------------
 √ certificate config is valid
-√ trust roots are using supported crypto algorithm
-√ trust roots are within their validity period
-√ trust roots are valid for at least 60 days
+√ trust anchors are using supported crypto algorithm
+√ trust anchors are within their validity period
+√ trust anchors are valid for at least 60 days
 √ issuer cert is using supported crypto algorithm
 √ issuer cert is within its validity period
 √ issuer cert is valid for at least 60 days
-√ issuer cert is issued by the trust root
+√ issuer cert is issued by the trust anchor
 
 linkerd-identity-data-plane
 ---------------------------