Repository: google/turbinia
Pull request #1091 (2022-07-21 16:03:01)

Celery k8s deployment scripts
* GKE Celery Deployment Docs and update to other docs
  *  install-gcp-pubsub -> originating from install.md, instructions for terraform/gcp instance setup.
  * install-gke-pubsub -> instructions for installing PubSub GKE.
  * install-gke-celery -> instructions for installing Celery GKE.
  * turbinia-local-stack -> no changes, but shows instructions for local setup.
* Celery k8s files and celery deployment + cleanup scripts
* Added new turbinia config `CLOUD_PROVIDER` to indicate whether it is a GCP or Local install.
* Added logic throughout Turbinia code where GCP library imports happen to check for `CLOUD_PROVIDER` before importing
* Changed `processors.google_cloud.GetLocalInstanceName` to grab `hostname` instead of `name` due to not being available in GKE metadata.
* Updated `RedisStateManager` with increased socket timeout and to keep socket alive to help prevent any connection timeouts.
* Updated `tcelery.setup` with `task_acks_late` to acknowledge the Task after it has executed in case of a crash. and `worker_cancel_long_running_tasks_on_connection_loss` to kill long-running tasks on connection loss.
* Added in `config.logger.py` a check for log path and if not to create one due to some issues with deployment and the path not existing at startup.
* Small tweaks to Turbinia config file with how Celery/Redis variables are passed
* Update `Kombu` handler to increase socket timeout and to keep socket alive to help prevent any connection timeouts.
* Disabled `--without-gossip` and `--without-mingle` flags for Celery workers given noise generating pings and not useful.
  * Also good ref:  https://stackoverflow.com/questions/55249197/what-are-the-consequences-of-disabling-gossip-mingle-and-heartbeat-for-celery-w



dfjxs: yaml files LGTM

Patch:
diff --git a/docs/user/gke-sre.md b/docs/user/gke-sre.md
new file mode 100644
index 0000000..3db7144
--- /dev/null
+++ b/docs/user/gke-sre.md
@@ -0,0 +1,310 @@
+# **GKE SRE Guide to Turbinia**
+
+## Introduction
+
+This document covers the Turbinia SRE guide for Google Cloud Kubernetes. It will
+cover topics to manage the Turbinia infrastructure in the Kubernetes environment
+and includes the Prometheus/Grafana monitoring stack.
+
+## Debugging Task Failures
+
+At times, Turbinia may report back some failures after processing some Evidence.
+Given that Turbinia Jobs and Tasks can be created to run third party tools,
+Turbinia can not anticipate all failures that may occur, especially with a third
+party tool. Here are some debugging steps you can take to further investigate
+these failures.
+
+- Refer to the [debugging documentation](debugging.md)
+  for steps on grabbing the status of a Request or Task that has failed.
+- If the debugging documentation doesn’t provide enough information to the Task
+  failure, you may also grab and review stderr logs for the Task that has failed.
+  - stderr logs can be found in the path specified in the Turbinia `OUTPUT_DIR`.
+    The directory containing all Task output can be identified in the directory format
+    `<REQUEST_ID>-<TASK_ID>-<TASK_NAME>`.
+  - Turbinia logs can be found in the path specified at `LOG_DIR`.
+- Determine whether the failure has occurred before by checking the Error
+  Reporting console, if `STACKDRIVER_TRACEBACK` was enabled in the Turbinia config.
+  All Turbinia exceptions will be logged to the console and can be helpful to
+  check to see if the Error has been seen before and whether or not it has been
+  acknowledged/tracked in an issue.
+- Determine whether the Task failure is being tracked in a Github issue. If the
+  failure occurred from a third party tool, then we’ll likely NOT have tracked
+  this since the issue would have to be raised with the third party tool rather
+  than Turbinia.
+- If the issue seems to be related to the third party tool, file a bug to the
+  associated repo else file one for the Turbinia team.
+
+## GKE Infrastructure
+
+### Preparation
+
+The GKE stack is managed with the [update-gke-infra.sh](https://github.com/google/turbinia/raw/master/k8s/tools/update-gke-infra.sh) management script. This script can be run from any workstation or cloud shell.
+Please follow the steps below on a workstation or cloud shell prior to running
+the script.
+
+- Clone the Turbinia repo or the update-gke-infra.sh script directly.
+- Install [Google Cloud SDK](https://cloud.google.com/sdk/docs/install), which
+  installs the gcloud and kubectl cli tool.
+- Authenticate with the Turbinia cloud project:
+  - `gcloud auth application-default login`
+- Connect to the cluster
+  - `gcloud container clusters get-credentials [cluster] --zone [zone] --project [project]`
+
+## Updating the Turbinia infrastructure
+
+The following section will cover how to make updates to the Turbinia
+configuration file, environment variables, and updating the Turbinia Docker
+image.
+
+### Update the Turbinia configuration
+
+The Turbinia configuration is base64 encoded as a ConfigMap value named
+`TURBINIA_CONF`. This is then read by the Turbinia Server and Workers as an
+environment variable. Any changes made to the configuration do NOT require a
+Server/Worker restart if using the `update-gke-infra.sh` as the script will
+automatically restart the pods through a `kubectl rollout`
+
+Please ensure you have the latest version of the configuration file before
+making any changes. The new configuration can be loaded into the Turbinia stack
+through the following command
+
+- `$ ./update-gke-infra.sh -c update-config -f [path-to-cleartext-config]`
+- Note: the script will automatically encode the config file passed in as base64
+
+### Update an environment variable
+
+The Turbinia stack sets some configuration parameters through Deployment files,
+one for the Turbinia Server and one for Workers. In order to update an
+environment variable, run the following command.
+
+- `$ ./update-gke-infra.sh -c update-config -k [env-variable-name] -v [env-variable-value]`
+
+### Updating the Turbinia Docker image
+
+Turbinia is currently built as a Docker image which runs in a containerd
+environment.
+
+#### Updating to latest
+
+When a new version of Turbinia is released, a production Docker image will be
+built for both the Server and Worker and tagged with the `latest` tag or a tag
+specifying the [release date](https://github.com/google/turbinia/releases).
+It is recommended to specify the latest release date tag (e.g. `20220701`) instead
+of the `latest` tag to prevent Worker pods from picking up a newer version than the rest of the
+environment as they get removed and re-created through auto scaling. Additionaly,
+an older release date can be specified if you'd like to rollback to a different
+version of Turbinia. These updates can be done through the commands below.
+
+- `$ ./update-gke-infra.sh -c change-image -t [tag]`
+
+## Scaling Turbinia
+
+### Scaling Turbinia Worker Pods
+
+Turbinia GKE automatically scales the number of Worker pods based on processing
+demand determined by the CPU utilization average across all pods. As demand
+increases, the number of pods scale up until the CPU utilization is below a
+determined threshold. Once processing is complete, the number of Worker pods
+will scale down. The current autoscaling policy is configured in the
+[turbinia-autoscale-cpu.yaml](https://github.com/google/turbinia/blob/master/k8s/gcp-pubsub/turbinia-autoscale-cpu.yaml)
+file.
+
+There is a default setting of 3 Worker pods to run at any given time with the
+ability to scale up to 50 Worker pods across all nodes in the GKE cluster.
+In order to update the minimum number of Worker pods running at a given time,
+update the `minReplicas` value with the desired number of pods. In order to update
+the max number of pods to scale, update the `maxReplicas` value with the desired
+number. These changes should be updated in the [turbinia-autoscale-cpu.yaml](https://github.com/google/turbinia/blob/master/k8s/gcp-pubsub/turbinia-autoscale-cpu.yaml)
+file then applied through the following command.
+
+- `$ kubectl replace -f turbinia-autoscale-cpu.yaml`
+
+### Scaling Turbinia Nodes
+
+Currently, Turbinia does not currently support the autoscaling of nodes in GKE.
+There is a default setting of 1 node to run in the GKE cluster. In order to
+update the minimum number of nodes running, update the `CLUSTER_NODE_SIZE` value
+in [.clusterconfig](https://github.com/google/turbinia/blob/master/k8s/tools/.clusterconfig)
+with the desired number of nodes.
+
+## Helpful K8s Commands
+
+In addition to using the update-gke-infra.sh script to manage the cluster, the
+kubectl CLI can come useful for running administrative commands against the
+cluster, to which you can find some useful commands below.
+A verbose cheatsheet can also be found [here](https://kubernetes.io/docs/reference/kubectl/cheatsheet/).
+
+- Authenticating to the cluster (run this before any other kubectl commands)
+
+  - `$ gcloud container clusters get-credentials [cluster-name] --zone [zone] --project [project-name]`
+
+- Get cluster events
+
+  - `$ kubectl get events`
+
+- Get Turbinia pods
+
+  - `$ kubectl get pods`
+
+- Get all pods (includes monitoring pods)
+
+  - `$ kubectl get pods -A`
+
+- Get all pods and associated nodes
+
+  - `$ kubectl get pods -A -o wide`
+
+- Get verbose related pod deployment status
+
+  - `$ kubectl describe pod [pod-name]`
+
+- Get all nodes
+
+  - `$ kubectl get nodes`
+
+* Get logs from specific pod
+
+  - `$ kubectl logs [pod-name]`
+
+- SSH into specific pod
+
+  - `$ kubectl exec —-stdin —-tty [pod-name] —- bash`
+
+- Execute command into specific pod
+
+  - `$ kubectl exec [pod-name] —- [command]`
+
+- Get Turbinia ConfigMap
+
+  - `$ kubectl get configmap turbinia-config -o json | jq '.data.TURBINIA_CONF' | xargs | base64 -d`
+
+- Apply k8s yaml file
+
+  - $ `kubectl apply -f [path-to-file]`
+
+- Replace a k8s yaml file (updates appropriate pods)
+
+  - $ `kubectl replace -f [path-to-file]`
+
+- Delete a pod
+
+  - $ `kubectl delete pod [pod-name]`
+
+- Force delete all pods
+
+  - `$ kubectl delete pods —-all —-force —-grace-period=0`
+
+- Get horizontal scaling numbers (hpa)
+
+  - `$ kubectl get hpa`
+
+- See how busy (cpu/mem) pods are
+
+  - `$ kubectl top pods`
+
+- See how busy (cpu/mem) nodes are
+
+  - `$ kubectl top nodes`
+
+## GKE Metrics and Monitoring
+
+In order to monitor the Turbinia infrastructure within Kubernetes,
+we are using the helm chart [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus)
+to deploy the Prometheus stack to the cluster. This simplifies the setup required
+and automatically deploys Prometheus, Grafana, and Alert Manager to the cluster
+through manifest files.
+
+The Turbinia Server and Workers are instrumented with Prometheus code and expose
+application metrics.
+
+- Service manifest files were created for both the Turbinia [Server](https://github.com/google/turbinia/blob/master/k8s/gcp-pubsub/turbinia-server-metrics-service.yaml) and [Worker](https://github.com/google/turbinia/blob/master/k8s/gcp-pubsub/turbinia-worker-metrics-service.yaml).
+- The files create two services named `turbinia-server-metrics` and `turbinia-worker-metrics` which expose port 9200 to
+  poll application metrics.
+- The Prometheus service, which is listening on port 9090 scrapes these services
+  for metrics.
+- Grafana pulls system and application metrics from Prometheus and displays
+  dashboards for both os and application metrics. Grafana is listening on port 3000.
+
+### Connecting to Prometheus instance
+
+In order to connect to the Prometheus instance, go to the cloud console and
+connect to the cluster using cloud shell. Then run the following command to port
+forward the Prometheus service.
+
+- `$ kubectl --namespace monitoring port-forward svc/prometheus-k8s 9090`
+
+Once port forwarding, on the top right of the cloud shell console next to
+“Open Editor” there is an option for “Web Preview”. Click on that then change
+the port to 9090. This should then connect you to the Prometheus instance.
+
+### Connecting to Grafana instance
+
+In order to connect to the Grafana instance, go to the cloud console and connect
+to the cluster using cloud shell. Then run the following command to port forward
+the Grafana service.
+
+- `$ kubectl --namespace monitoring port-forward svc/grafana 11111:3000`
+
+Once port forwarding, on the top right of the cloud shell console next to
+“Open Editor” there is an option for “Web Preview”. Click on that then change
+the port to 11111. This should then connect you to the Grafana instance.
+
+## Grafana and Prometheus config
+
+This section covers how to update and manage the Grafana and Prometheus instances
+for adding new rules and updating the dashboard.
+
+### Importing a new dashboard into Grafana
+
+- Login to the Grafana instance
+- Click the “+” sign on the left sidebar and then select “import”.
+- Then copy/paste the json file from the dashboard you want to import and click “Load”.
+
+### Exporting a dashboard from Grafana
+
+- Login to Grafana
+- Navigate to the dashboard you’d like to export
+- From the dashboard, select the “dashboard Setting” on the upper right corner
+- Click on “JSON Model” and copy the contents of the textbox.
+- To import this to another dashboard, follow the steps outlined in importing a new dashboard.
+
+### Updating the Prometheus Config
+
+To update Prometheus with any additional configuration options, take the
+following steps.
+
+- Clone the github repo [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus)
+  locally.
+- Once cloned, navigate to the [manifests/prometheus-prometheus.yaml](https://github.com/prometheus-operator/kube-prometheus/blob/main/manifests/prometheus-prometheus.yaml) file and make any necessary changes.
+- Also ensure that the additional scrape config is added back into the bottom of the file as it’s required for Prometheus to query for Turbinia metrics.
+
+  ```
+  additionalScrapeConfigs:
+    name: additional-scrape-configs
+    key: prometheus-additional.yaml
+  ```
+
+* Once done, replace the Prometheus config file by running
+  - `$ kubectl --namespace monitoring replace -f manifests/prometheus-prometheus.yaml`
+  - Note: The updates should automatically take place
+
+### Updating Prometheus Rules
+
+To update the Prometheus rules, take the following steps.
+
+- Create or update an existing rule file. Please see [here](https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) for great tips on writing recording rules.
+- Once your rule has been created, append the rule to the
+  turbinia-custom-rules.yaml file following a similar format as the other rules.
+  ```
+  - name: [rule-name]
+  rules:
+  # Comment describing rule
+  - record: [record-value]
+    expr: [expr-value]
+  ```
+- Once added into the file, update the monitoring rules by running the following
+  - `$ kubectl --namespace monitoring replace -f turbinia-custom-rules.yaml`
+
+* Verify that the changes have taken place by navigating to the Prometheus
+  instance after a few minutes then going to Status -> Rules and searching for the
+  name of your newly created rule.
diff --git a/docs/user/index.rst b/docs/user/index.rst
index ffe5e33..733967d 100644
--- a/docs/user/index.rst
+++ b/docs/user/index.rst
@@ -4,11 +4,12 @@ User documentation
 
 .. toctree::
    :maxdepth: 3
-
    install
-   install-manual
-   install-gke-manual
-   install-gke
+   install-gke-celery
+   install-gke-pubsub
+   install-gke-monitoring
+   install-gcp-pubsub
+   gke-sre
    how-it-works
    turbinia-local-stack
    using-docker
diff --git a/docs/user/install-gcp-pubsub.md b/docs/user/install-gcp-pubsub.md
new file mode 100644
index 0000000..9807202
--- /dev/null
+++ b/docs/user/install-gcp-pubsub.md
@@ -0,0 +1,157 @@
+# **Turbinia GCP Installation Instructions**
+
+## Overview
+
+These instructions cover the PubSub installation of Turbinia using
+[Google Cloud Platform](https://cloud.google.com). This uses
+[terraform configs](https://github.com/forseti-security/osdfir-infrastructure)
+that are part of the [Forseti Security repository](https://github.com/forseti-security)
+to automate deployment of Turbinia into an existing GCP Project.
+
+## Installation
+
+The following steps can be performed on any Linux machine (Ubuntu 20.04
+recommended), and [Cloud Shell](https://cloud.google.com/shell/) is one easy way
+to get a shell with access to your GCP resources.
+
+### GCP Project Setup
+
+- Create or select a Google Cloud Platform project in the
+  [Google Cloud Console](https://console.cloud.google.com).
+- Determine which GCP zone and region that you wish to deploy Turbinia into.
+  Note that one of the GCP dependencies is Cloud Functions, and that only
+  works in certain regions, so you will need to deploy in one of
+  [the supported regions](https://cloud.google.com/functions/docs/locations).
+- Install
+  [google-cloud-sdk](https://cloud.google.com/sdk/docs/quickstart-linux).
+  - Note: If you are doing this from cloud shell you shouldn't need this
+    step.
+- Run `gcloud auth login` to authenticate. This may require you to copy/paste
+  url to browser.
+- Run `gcloud auth application-default login`
+
+### Deploy Turbinia
+
+- Download the
+  [Terraform CLI from here](https://www.terraform.io/downloads.html).
+- Clone the Forseti Security repository and change to the path containing the
+  configs
+  - `git clone https://github.com/forseti-security/osdfir-infrastructure/`
+  - `cd osdfir-infrastructure`
+- Configuration
+
+  - By default this will create one Turbinia server instance and one worker
+    instance. If you want to change the number of workers, edit the
+    `modules/turbinia/variables.tf` file and set the `turbinia_worker_count`
+    variable to the number of workers you want to deploy.
+  - To adjust the GCP zone and region you want to run Turbinia in, edit the
+    `modules/turbinia/variables.tf` file and change the `gcp_zone` and
+    `gcp_region` variables as appropriate to reflect your GCP project's
+    zone and region.
+  - If you want to use Docker to run Turbinia tasks, please follow the
+    instructions [here](using-docker.md) to enable Docker.
+  - Running the following commands will leave some state information under
+    the current directory, so if you wish to continue to manage the number
+    of workers via Terraform you should keep this directory for later use.
+    Alternatively, if you wish to store this information in GCS instead, you
+    can edit `main.tf` and change the `bucket` parameter to the GCS bucket
+    you wish to keep this state information in. See the
+    [Terraform documentation](https://www.terraform.io/docs/commands/index.html)
+    for more information.
+  - The current configuration does not enable alert notifications by default.
+    Please see [here](#grafana-smtp-setup) for instructions.
+  - If you are running multiple workers on a given host and within containers, ensure
+    that you are mapping the host `OUTPUT_DIR` path specified in the configuration file
+    `.turbiniarc` to the containers so that they can properly update the `RESOURCE_STATE_FILE`.
+
+- Initialize terraform and apply the configuration
+  - `./deploy.sh --no-timesketch`
+    - If the `--no-timesketch` parameter is not supplied, Terraform will also
+      create a [Timesketch](http://timesketch.org) instance in the same
+      project, and this can be configured to ingest Turbinia timeline
+      output and report data. See the
+      [Documentation on this](https://github.com/forseti-security/osdfir-infrastructure)
+      for more details.
+    - When prompted for the project name, enter the project you selected
+      during setup.
+
+This should result in the appropriate cloud services being enabled and
+configured and GCE instances for the server and the worker(s) being started and
+configured. The Turbinia configuration file will be deployed on these instances
+as `etc/turbinia/turbinia.conf`. If you later want to increase the number of
+workers, you can edit the `turbinia/variables.tf` file mentioned above and
+re-run `terraform apply`
+To use Turbinia you can use the virtual environment that was setup by
+the `deploy.sh` script.To activate the virtual environment, run the following
+command `source ~/turbinia/bin/activate` and then use `turbiniactl`. For more
+information on how to use Turbinia please visit [the user manual](https://github.com/google/turbinia).
+
+### Client configuration (optional)
+
+If you want to use the command line tool, you can SSH into the server and run
+`turbiniactl` from there. The `turbiniactl` command can be used to submit
+Evidence for processing or see the status of existing and previous processing
+requests. If you'd prefer to use turbiniactl on a different machine, follow the
+following instructions to configure the client. The instructions are based on
+using Ubuntu 20.04, though other versions of Linux should be compatible.
+
+- Follow the steps from GCP Project setup above to install the SDK and
+  authenticate with gcloud.
+- Install some python tooling:
+  - `apt-get install python3-pip python3-wheel`
+- Install the Turbinia client.
+  - Note: You may want to install this into a virtual-environment with
+    [venv](https://docs.python.org/3.7/library/venv.html) or
+    [pipenv](https://pipenv.pypa.io/en/latest/)) to reduce potential
+    dependency conflicts and isolate these packages into their own
+    environment.
+  - `pip3 --user install turbinia`
+- If running on the same machine you deployed Turbinia from, you can generate
+  the config with terraform
+  - `terraform output turbinia-config > ~/.turbiniarc`
+- Otherwise, if you are running from a different machine you'll need to copy
+  the Turbinia config from the original machine, or from the Turbinia server
+  from `/etc/turbinia/turbinia.conf`.
+
+### Grafana SMTP Setup
+
+If you want to receive alert notifications from Grafana, you'll need to setup a SMTP server for Grafana. To configure a SMTP server, you need to add the following environment variables to `Grafana` `env` section in `osdfir-infrastructure/modules/monitoring/main.tf`
+
+```
+      {
+        name = "GF_SMTP_ENABLED"
+        value = "true"
+      }, {
+        name = "GF_SMTP_HOST"
+        value = "smtp.gmail.com:465" # Replace this if you're not using gmail
+      }, {
+        name = "GF_SMTP_USER"
+        value = "<EMAIL ADDRESS HERE>"
+      }, {
+        name = "GF_SMTP_PASSWORD"
+        value = "<PASSWORD>"
+      }, {
+        name = "GF_SMTP_SKIP_VERIFY"
+        value = "true"
+      }, {
+        name = "GF_SMTP_FROM_ADDRESS"
+        value = "<EMAIL ADDRESS THAT SHOWS AS THE SENDER>"
+      }
+
+```
+
+---
+
+> **NOTE**
+
+> By default Gmail does not allow [less secure apps](https://support.google.com/accounts/answer/6010255) to authenticate and send emails. For that reason, you'll need to allow less secure apps to access the provided Gmail account.
+
+---
+
+Once completed:
+
+- login to the Grafana Dashboard.
+- Select Alerting and choose "Notification channels".
+- Fill the required fields and add the email addresses that will receive notification.
+- Click "Test" to test your SMTP setup.
+- Once everything is working, click "Save" to save the notification channel.
diff --git a/docs/user/install-gke-celery.md b/docs/user/install-gke-celery.md
new file mode 100644
index 0000000..6b45136
--- /dev/null
+++ b/docs/user/install-gke-celery.md
@@ -0,0 +1,48 @@
+# Turbinia GKE Celery Installation Instructions
+
+## **Introduction**
+
+These instructions cover the Redis installation of Turbinia using [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine). However, development is still in progress and we
+recommend you use the [GKE PubSub Installation](install-gke-pubsub.md) for cloud installations.
+
+Installing into GKE allows Turbinia Workers to scale based on processing demand. Currently, this is done through scaling on CPU utilization, which is determined when available Turbinia Workers process Tasks and reach a pre-defined CPU threshold. The GKE architecture closely resembles the [cloud architecture](how-it-works.md) with GKE being used to scale Turbinia Woker pods.
+
+All steps in this document are required for getting Turbinia running on GKE.
+
+### **Prerequisites**
+
+GKE is only supported for Google Cloud so a Google Cloud Project is required to work from.
+
+## **Installation**
+
+Please follow these steps for deploying Turbinia to GKE. Ensure that the `.clusterconfig` config file has been updated appropriately.
+
+### **Turbinia GKE Deployment**
+
+**Follow these steps to deploy Turbinia to GKE.**
+
+- Create or select a Google Cloud Platform project in the
+  [Google Cloud Console](https://console.cloud.google.com).
+- Determine which GCP zone and region that you wish to deploy Turbinia into.
+- Review the `.clusterconfig` config file and please update any of the default values if necessary based on requirements.
+- Deploy Turbinia through the following command
+  - `./k8s/tools/deploy-celery-gke.sh`
+- The deployment script will automatically enable GCP APIs, create the cluster and GCP resources then deploy Turbinia to the cluster. At the end of the run, you should have a fully functioning Turbinia environment within GKE to use.
+
+**Follow these steps to destroy the Turbinia GKE environment.**
+
+- Run the following command if you'd like to destroy the Turbinia GKE environment.
+  - `./k8s/tools/destroy-celery-gke.sh`
+  - **Note this will delete the Turbinia cluster including all processed output and log files as well as associated GCP resources**
+
+### **Making processing requests in GKE**
+
+- You can either make requests via setting up a local `turbiniactl` client or through connecting to the server through the following steps.
+- Connect to cluster through `gcloud container clusters get-credentials <CLUSTER_NAME> --zone <ZONE> --project <PROJECT_NAME>`.
+- Use `kubectl get pods` to get a list of running pods.
+- Identify the pod named `turbinia-server-*` and exec into it via `kubectl exec --stdin --tty [CONTAINER-NAME] -- bash`
+- Use `turbiniactl` to kick off a request to process evidence.
+
+## **Monitoring Installation**
+
+Turbinia GKE has the capability to be monitored through Prometheus and Grafana. Please follow the steps outlined under the Monitoring Installation section [here](install-gke-monitoring.md).
diff --git a/docs/user/install-gke-manual.md b/docs/user/install-gke-manual.md
deleted file mode 100644
index a79dd0f..0000000
--- a/docs/user/install-gke-manual.md
+++ /dev/null
@@ -1,203 +0,0 @@
-# Turbinia GKE Manual Installation Instructions
-
-## **Introduction**
-
-These instructions cover how to manually install Turbinia using [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine).  It is recommended to use the [Turbinia GKE Quick Installation Instructions](install-gke.md) for an easier and more automated deployment.
-
-Installing Turbinia in GKE allows Turbinia Workers to scale based on processing demand. Currently, this is done through scaling on CPU utilization, which is determined when available Turbinia Workers process Tasks and reach a pre-defined CPU threshold. The GKE architecture closely resembles the [cloud architecture](how-it-works.md) with GKE being used to scale Turbinia Woker pods.
-
-All steps in this document are required for getting Turbinia running on GKE. Please ensure you have followed the GCP setup prior to configuring the GKE cluster as it's required for Turbinia components to properly function together.
-
-### **Prerequisites**
-
-GKE is only supported for Google Cloud so a Google Cloud Project is required to work from. Additionally, all GCP components specified below must be enabled so please follow the GCP steps outlined prior to setting up GKE.
-
-## **Installation**
-
-Please follow these steps for configuring Turbinia for GCP use and then running it within GKE. Ensure that the `.turbiniarc` config file has been updated appropriately. 
-
-### **GCP Setup**
-
-**Follow these steps prior to configuring GKE for Turbinia.**
-
-*   Create or select a Google Cloud Platform project in the
-    [Google Cloud Console](https://console.cloud.google.com).
-*   Determine which GCP zone and region that you wish to deploy Turbinia into.
-    Note that one of the GCP dependencies is Cloud Functions, and that only
-    works in certain regions, so you will need to deploy in one of
-    [the supported regions](https://cloud.google.com/functions/docs/locations).
-*   Enable
-    [Cloud Functions](https://console.cloud.google.com/apis/library/cloudfunctions.googleapis.com).
-*   Follow the
-    [instructions](https://cloud.google.com/pubsub/docs/quickstart-console) to:
-    *   Enable
-        [Cloud Pub/Sub](https://console.cloud.google.com/apis/library/pubsub.googleapis.com)
-    *   Create a new Pub/Sub topic and subscription (pull type with 600s
-        timeout). These can use the same base name (the part after `topics/` and
-        `subscription/` in the paths).
-    *   Please take a note of the topic name for the configuration steps, as
-        this is what you will set the `PUBSUB_TOPIC` config variable to.
-*   Enable
-    [Cloud Datastore](https://console.cloud.google.com/apis/api/datastore.googleapis.com)
-    *   Go to Datastore in the cloud console
-    *   Hit the `Create Entity` button
-    *   Select the same region that you selected in the previous steps. No need
-        to create any Entities after selecting your region
-*   [Create a new GCS bucket](https://cloud.google.com/storage/docs/creating-buckets)
-    and take note of the bucket name as this will be referenced later by the
-    `GCS_OUTPUT_PATH` variable.
-*   Deploy Cloud Functions `cd <git clone path>/tools/gcf_init && ./deploy_gcf.py`
-* Enable [Cloud Filestore](https://console.cloud.google.com/filestore)
-    * Go to Filestore in the cloud console
-    * Hit the `Create Instance` button
-    * Set the `Instance ID` and `File Share Name` to the default name `output`.
-        * Note: The name can be changed however you must update the appropriate sections in the k8s Deployment files with the newly chosen name. Please see the `GKE Setup` section for more details.
-    * Select the same region and zone selected in the previous steps.
-    * After the Filestore instance has been created, keep a note of the IP address for later use.
-
-### **GKE Setup**
-* Enable the [Kubernetes Engine API](https://console.cloud.google.com/apis/api/container.googleapis.com/overview).
-* Create or select a Google Kubernetes Engine cluster in the
-    [Google Cloud Console](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-zonal-cluster#console).
-  * Choose the GKE Standard type.
-  * Choose any name for the cluster.
-  * The Zone should match the GCP Zone configuration (e.g. wherever Zone cloud functions & Datastore was deployed to).
-  * Choose the number of nodes. The minimum requirement being 3 nodes and recommended being 5 nodes.
-  * Change the machine type. The minimum recommendation would be to use a machine type of e2-standard-32.
-  * In the Node Pools -> Security tab, change access scopes to "Allow full access to all cloud APIs"
-* Alternatively, a GKE cluster can be created via [gcloud](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-zonal-cluster#gcloud).
-```
-gcloud container clusters create CLUSTER_NAME \
-    --release-channel None \
-    --zone COMPUTE_ZONE \ # Should match GCS/GCP Zones.
-    --node-locations COMPUTE_ZONE # Should match GCS/GCP Zones.
-    --num_nodes 3 \ # Change based on processing requirements.
-    --machine-type e2-medium # Change based on processing requirements.
-    --scopes "https://www.googleapis.com/auth/cloud-platform" 
-```
-### **Turbinia GKE Deployment**
-* Connect to cluster through `gcloud container clusters get-credentials <CLUSTER_NAME> --zone <ZONE> --project <PROJECT_NAME>`.
-* Clone the latest Turbinia branch and `cd <git clone path>/k8s/gcp-pubsub`.
-* Ensure that the zone and region in the Turbinia config file are equal to the zone and region you created your k8s cluster in.
-* The `image` variable can be optionally changed in the `turbinia-worker.yaml` and `turbinia-server.yaml` files e.g expiremental/dev to chose the docker images used during deployment.
-* In the `turbinia-worker.yaml` file, ensure that the path in the volume labeled `lockfolder` matches the Turbinia config variable `TMP_RESOURCE_DIR`.
-* If the Filestore `Instance ID` and `File share name` have a different name than the default name `output`, update `turbinia-worker.yaml`, `turbinia-server.yaml`, `turbinia-output-claim-filestore.yaml`, and `turbinia-output-filestore.yaml` by searching for the string `output` and replacing it with the custom name. Skip this step if the default name was used.
-* To have all logs go to the central location, update the `LOG_DIR` variable in the `.turbiniarc` config file to the default Filestore path `/mnt/output` or if configured differently, to the custom path.
-* In `turbinia-output-filestore.yaml`, update `<IP_ADDRESS>` to the Filestore IP address.
-* If the Filestore instance size is greater than 1 TB, update the `storage` sections in `turbinia-output-claim-filestore.yaml` and `turbinia-output-filestore.yaml` with the appropriate size.
-* Ensure that the `.turbiniarc` config file has been properly configured with required GCP variables.
-* Deploy the Turbinia infrastructure by executing `./setup-pubsub.sh <PATH TO CONFIG>`. 
-* The Turbinia infrastructure can be destroyed by executing `./destroy-pubsub.sh`.
-
-### **Making processing requests in GKE**
-* You can either make requests via setting up a local `turbiniactl` client using pip install turbinia in eg. a virtualenv or through connecting to the server through the following steps.
-* Connect to cluster through `gcloud container clusters get-credentials <CLUSTER_NAME> --zone <ZONE> --project <PROJECT_NAME>`.
-* Use `kubectl get pods` to get a list of running pods.
-* Identify the pod named `turbinia-server-*` and exec into it via `kubectl exec --stdin --tty [CONTAINER-NAME] -- bash`
-* Use `turbiniactl` to kick off a request to process evidence.
-
-## **Monitoring Installation**
-Turbinia GKE has the capability to be monitored through Prometheus and Grafana. Please follow these steps for configuring Turbinia for monitoring and ensure that the `.turbiniarc` config file has been updated appropriately. 
-
-### Application Metrics
-In order to receive Turbinia application metrics, you'll need to adjust the following variables in the `.turbinarc` config file. 
-```
-PROMETHEUS_ENABLED = True
-PROMETHEUS_ADDR = '0.0.0.0'
-PROMETHEUS_PORT = 9200
-```
-Please ensure `PROMETHEUS_ENABLED` is set to `True` and that the `PROMETHEUS_PORT` matches the `prometheus.io/port` section in the `turbinia-worker.yaml` and `turbinia-server.yaml` as well as matching ports in the `turbinia-server-metrics-service.yaml` and `turbinia-worker-metrics-service.yaml` GKE deployment files.
-
-### Deploying Prometheus
-In this deployment method, we are using [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) to deploy the Prometheus stack to the cluster. This simplifies the setup required and automatically deploys Prometheus, Grafana, and Alert Manager to the cluster through manifest files. Before proceeding with the setup, please ensure you are connected to the cluster with Turbinia deployed and can run commands via `kubectl`, then proceed to the following steps to configure Prometheus with Turbinia.
-
-* Clone the github repo [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) locally. Please ensure that the branch  cloned is compatible with your Kubernetes cluster version else you may run into issues. Please see the [Compatibility Matrix](https://github.com/prometheus-operator/kube-prometheus) section of the repo for more details.
-* Once cloned, run the following commands to deploy the stack
-    * `kubectl create -f manifests/setup`
-    * `kubectl create -f manifests/`
-* Create a secret from the file `prometheus-additional.yaml` located in the Turbinia folder.
-    * `kubectl create secret generic additional-scrape-configs --from-file=monitoring/k8s/prometheus/prometheus-additional.yaml --dry-run=client -oyaml > additional-scrape-configs.yaml`
-* You will then need to update the `prometheus-prometheus.yaml` file located in the `kube-prometheus/manifests` folder with this extra scrape config
-    ```
-    additionalScrapeConfigs:
-        name: additional-scrape-configs
-        key: prometheus-additional.yaml
-    ```
-* Once complete apply the changes made through the following commands
-    * `kubectl -n monitoring apply -f additional-scrape-configs.yaml`
-    * `kubectl -n monitoring apply -f manifests/prometheus-prometheus.yaml`
-* To import Turbinia custom rules, run the `gen-yaml.sh` script from the same directory its located
-    * `cd monitoring/k8s && ./gen-yaml.sh`
-* Then apply the `turbinia-custom-rules.yaml` file 
-    * `kubectl -n monitoring apply -f monitoring/k8s/prometheus/turbinia-custom-rules.yaml`
-
-### Testing Prometheus Deployment
-* Test that the changes were properly made by connecting to the Prometheus console and searching for `turbinia`. If related metrics pop up in the search bar, then Turbinia metrics are properly being ingested by Prometheus. You can also check to see if the Turbinia custom rules have been applied by navigating to Status -> Rules then searching for one of the custom rule names. To connect to the Prometheus console, run the following command
-    * `kubectl -n monitoring port-forward svc/prometheus-k8s 9090`
-
-* To delete the monitoring stack, cd into the `kube-prometheus` directory and run the following command.
-    * `kubectl delete --ignore-not-found=true -f manifests/ -f manifests/setup`
-
-### Deploying Grafana
-Before proceeding to the Grafana setup, please ensure that you have followed all the steps outlined in the **Testing Prometheus Deployment** section.
-* Clone the github repo [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) locally. 
-* You will then need to update `manifests/grafana-deployment.yaml` file, first by updating the `volumeMounts` section with the following `mountPaths`
-    ```
-    - mountPath: /grafana-dashboard-definitions/0/turbinia-healthcheck-metrics
-      name: turbinia-healthcheck-metrics
-      readOnly: false
-    - mountPath: /grafana-dashboard-definitions/0/turbinia-application-metrics
-      name: turbinia-application-metrics
-      readOnly: false
-    ```  
-* Then by updating the `volumes` section with the following `configMaps`
-    ```
-    - configMap:
-        name: turbinia-application-metrics
-      name: turbinia-application-metrics
-    - configMap:
-        name: turbinia-healthcheck-metrics
-      name: turbinia-healthcheck-metrics
-    ```
-* Once complete, apply the changes through
-  * `kubectl -n monitoring apply -f manifests/grafana-deployment.yaml`
-* To get the Turbinia Application & Healthcheck dashboard to show, first run the `gen.yaml.sh` if haven't done so already in the setting up Prometheus section.
-    * `cd monitoring/k8s && ./gen-yaml.sh`
-* Then apply the dashboards to the monitoring namespace.
-    * `kubectl -n monitoring apply -f monitoring/k8s/grafana`
-* To connect to the Grafana dashboard, run the following command
-    * ```kubectl -n monitoring port-forward svc/grafana 11111:3000```
-### Email Notifications
-
-If you want to receive alert notifications from Grafana, you'll need to setup a SMTP server for Grafana. To configure a SMTP server, you need to add the following environment variables to the `env` section of the `manifests/grafana-deployment.yaml` file.
-
-```
-- name: GF_SMTP_ENABLED
-  value: "true"
-- name: GF_SMTP_HOST
-  value: "smtp.gmail.com:465" #Replace this if you're not using gmail
-- name: GF_SMTP_USER
-  value: "<EMAIL_ADDRESS_HERE>"
-- name: GF_SMTP_PASSWORD
-  value: "<PASSWORD>"
-- name: GF_SMTP_SKIP_VERIFY
-  value: "true"
-- name: GF_SMTP_FROM_ADDRESS
-  value: "<EMAIL ADDRESS THAT SHOWS AS THE SENDER>"
-```
-Then apply the changes through the following command
-* `kubectl -n monitoring apply -f manifests/grafana-deployment.yaml` 
-
----
-> **NOTE**
-
-> By default Gmail does not allow [less secure apps](https://support.google.com/accounts/answer/6010255) to authenticate and send emails. For that reason, you'll need to allow less secure apps to access the provided Gmail account.
-
----
-
-Once completed:
- - login to the Grafana Dashboard.
- - Select Alerting and choose "Notification channels".
- - Fill the required fields and add the email addresses that will receive notification.
- - Click "Test" to test your SMTP setup.
- - Once everything is working, click "Save" to save the notification channel.
diff --git a/docs/user/install-gke-monitoring.md b/docs/user/install-gke-monitoring.md
new file mode 100644
index 0000000..f77af75
--- /dev/null
+++ b/docs/user/install-gke-monitoring.md
@@ -0,0 +1,119 @@
+## **Monitoring Installation**
+
+Turbinia GKE has the capability to be monitored through Prometheus and Grafana. Please follow these steps for configuring Turbinia for monitoring and ensure that the `.turbiniarc` config file has been updated appropriately.
+
+### Application Metrics
+
+In order to receive Turbinia application metrics, you'll need to adjust the following variables in the `.turbinarc` config file.
+
+```
+PROMETHEUS_ENABLED = True
+PROMETHEUS_ADDR = '0.0.0.0'
+PROMETHEUS_PORT = 9200
+```
+
+Please ensure `PROMETHEUS_ENABLED` is set to `True` and that the `PROMETHEUS_PORT` matches the `prometheus.io/port` section in the `turbinia-worker.yaml` and `turbinia-server.yaml` as well as matching ports in the `turbinia-server-metrics-service.yaml` and `turbinia-worker-metrics-service.yaml` GKE deployment files.
+
+### Deploying Prometheus
+
+In this deployment method, we are using [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) to deploy the Prometheus stack to the cluster. This simplifies the setup required and automatically deploys Prometheus, Grafana, and Alert Manager to the cluster through manifest files. Before proceeding with the setup, please ensure you are connected to the cluster with Turbinia deployed and can run commands via `kubectl`, then proceed to the following steps to configure Prometheus with Turbinia.
+
+- Clone the github repo [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) locally. Please ensure that the branch cloned is compatible with your Kubernetes cluster version else you may run into issues. Please see the [Compatibility Matrix](https://github.com/prometheus-operator/kube-prometheus) section of the repo for more details.
+- Once cloned, run the following commands to deploy the stack
+  - `kubectl create -f manifests/setup`
+  - `kubectl create -f manifests/`
+- Create a secret from the file `prometheus-additional.yaml` located in the Turbinia folder.
+  - `kubectl create secret generic additional-scrape-configs --from-file=monitoring/k8s/prometheus/prometheus-additional.yaml --dry-run=client -oyaml > additional-scrape-configs.yaml`
+- You will then need to update the `prometheus-prometheus.yaml` file located in the `kube-prometheus/manifests` folder with this extra scrape config
+  ```
+  additionalScrapeConfigs:
+      name: additional-scrape-configs
+      key: prometheus-additional.yaml
+  ```
+- Once complete apply the changes made through the following commands
+  - `kubectl -n monitoring apply -f additional-scrape-configs.yaml`
+  - `kubectl -n monitoring apply -f manifests/prometheus-prometheus.yaml`
+- To import Turbinia custom rules, run the `gen-yaml.sh` script from the same directory its located
+  - `cd monitoring/k8s && ./gen-yaml.sh`
+- Then apply the `turbinia-custom-rules.yaml` file
+  - `kubectl -n monitoring apply -f monitoring/k8s/prometheus/turbinia-custom-rules.yaml`
+
+### Testing Prometheus Deployment
+
+- Test that the changes were properly made by connecting to the Prometheus console and searching for `turbinia`. If related metrics pop up in the search bar, then Turbinia metrics are properly being ingested by Prometheus. You can also check to see if the Turbinia custom rules have been applied by navigating to Status -> Rules then searching for one of the custom rule names. To connect to the Prometheus console, run the following command
+
+  - `kubectl -n monitoring port-forward svc/prometheus-k8s 9090`
+
+- To delete the monitoring stack, cd into the `kube-prometheus` directory and run the following command.
+  - `kubectl delete --ignore-not-found=true -f manifests/ -f manifests/setup`
+
+### Deploying Grafana
+
+Before proceeding to the Grafana setup, please ensure that you have followed all the steps outlined in the **Testing Prometheus Deployment** section.
+
+- Clone the github repo [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus) locally.
+- You will then need to update `manifests/grafana-deployment.yaml` file, first by updating the `volumeMounts` section with the following `mountPaths`
+  ```
+  - mountPath: /grafana-dashboard-definitions/0/turbinia-healthcheck-metrics
+    name: turbinia-healthcheck-metrics
+    readOnly: false
+  - mountPath: /grafana-dashboard-definitions/0/turbinia-application-metrics
+    name: turbinia-application-metrics
+    readOnly: false
+  ```
+- Then by updating the `volumes` section with the following `configMaps`
+  ```
+  - configMap:
+      name: turbinia-application-metrics
+    name: turbinia-application-metrics
+  - configMap:
+      name: turbinia-healthcheck-metrics
+    name: turbinia-healthcheck-metrics
+  ```
+- Once complete, apply the changes through
+  - `kubectl -n monitoring apply -f manifests/grafana-deployment.yaml`
+- To get the Turbinia Application & Healthcheck dashboard to show, first run the `gen.yaml.sh` if haven't done so already in the setting up Prometheus section.
+  - `cd monitoring/k8s && ./gen-yaml.sh`
+- Then apply the dashboards to the monitoring namespace.
+  - `kubectl -n monitoring apply -f monitoring/k8s/grafana`
+- To connect to the Grafana dashboard, run the following command
+  - `kubectl -n monitoring port-forward svc/grafana 11111:3000`
+
+### Email Notifications
+
+If you want to receive alert notifications from Grafana, you'll need to setup a SMTP server for Grafana. To configure a SMTP server, you need to add the following environment variables to the `env` section of the `manifests/grafana-deployment.yaml` file.
+
+```
+- name: GF_SMTP_ENABLED
+  value: "true"
+- name: GF_SMTP_HOST
+  value: "smtp.gmail.com:465" #Replace this if you're not using gmail
+- name: GF_SMTP_USER
+  value: "<EMAIL_ADDRESS_HERE>"
+- name: GF_SMTP_PASSWORD
+  value: "<PASSWORD>"
+- name: GF_SMTP_SKIP_VERIFY
+  value: "true"
+- name: GF_SMTP_FROM_ADDRESS
+  value: "<EMAIL ADDRESS THAT SHOWS AS THE SENDER>"
+```
+
+Then apply the changes through the following command
+
+- `kubectl -n monitoring apply -f manifests/grafana-deployment.yaml`
+
+---
+
+> **NOTE**
+
+> By default Gmail does not allow [less secure apps](https://support.google.com/accounts/answer/6010255) to authenticate and send emails. For that reason, you'll need to allow less secure apps to access the provided Gmail account.
+
+---
+
+Once completed:
+
+- login to the Grafana Dashboard.
+- Select Alerting and choose "Notification channels".
+- Fill the required fields and add the email addresses that will receive notification.
+- Click "Test" to test your SMTP setup.
+- Once everything is working, click "Save" to save the notification channel.
diff --git a/docs/user/install-gke-pubsub.md b/docs/user/install-gke-pubsub.md
new file mode 100644
index 0000000..8e673aa
--- /dev/null
+++ b/docs/user/install-gke-pubsub.md
@@ -0,0 +1,47 @@
+# Turbinia GKE Quick Installation Instructions
+
+## **Introduction**
+
+These instructions cover the PubSub installation Turbinia using [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine).
+
+Installing into GKE allows Turbinia Workers to scale based on processing demand. Currently, this is done through scaling on CPU utilization, which is determined when available Turbinia Workers process Tasks and reach a pre-defined CPU threshold. The GKE architecture closely resembles the [cloud architecture](how-it-works.md) with GKE being used to scale Turbinia Woker pods.
+
+All steps in this document are required for getting Turbinia running on GKE.
+
+### **Prerequisites**
+
+GKE is only supported for Google Cloud so a Google Cloud Project is required to work from.
+
+## **Installation**
+
+Please follow these steps for deploying Turbinia to GKE. Ensure that the `.clusterconfig` config file has been updated appropriately.
+
+### **Turbinia GKE Deployment**
+
+**Follow these steps to deploy Turbinia to GKE.**
+
+- Create or select a Google Cloud Platform project in the
+  [Google Cloud Console](https://console.cloud.google.com).
+- Determine which GCP zone and region that you wish to deploy Turbinia into.
+  Note that one of the GCP dependencies is Cloud Functions, and that only
+  works in certain regions, so you will need to deploy in one of
+  [the supported regions](https://cloud.google.com/functions/docs/locations).
+- Review the `.clusterconfig` config file and please update any of the default values if necessary based on requirements.
+- Deploy Turbinia through the following command
+  - `./k8s/tools/deploy-pubsub-gke.sh`
+- The deployment script will automatically enable GCP APIs, create the cluster and GCP resources then deploy Turbinia to the cluster. At the end of the run, you should have a fully functioning Turbinia environment within GKE to use.
+- Run the following command if you'd like to cleanup the newly created Turbinia environment
+  - `./k8s/tools/cleanup-pubsub-gke.sh`
+  - **Note this will delete the Turbinia cluster including all processed output and log files as well as associated GCP resources**
+
+### **Making processing requests in GKE**
+
+- You can either make requests via setting up a local `turbiniactl` client or through connecting to the server through the following steps.
+- Connect to cluster through `gcloud container clusters get-credentials <CLUSTER_NAME> --zone <ZONE> --project <PROJECT_NAME>`.
+- Use `kubectl get pods` to get a list of running pods.
+- Identify the pod named `turbinia-server-*` and exec into it via `kubectl exec --stdin --tty [CONTAINER-NAME] -- bash`
+- Use `turbiniactl` to kick off a request to process evidence.
+
+## **Monitoring Installation**
+
+Turbinia GKE has the capability to be monitored through Prometheus and Grafana. Please follow the steps outlined under the Monitoring Installation section [here](install-gke-monitoring.md).
diff --git a/docs/user/install-gke.md b/docs/user/install-gke.md
deleted file mode 100644
index 4dfdc0f..0000000
--- a/docs/user/install-gke.md
+++ /dev/null
@@ -1,43 +0,0 @@
-# Turbinia GKE Quick Installation Instructions
-
-## **Introduction**
-
-These instructions cover the recommended quick installation Turbinia using [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine). For the longer manual deployment, see the [Turbinia GKE Manual Installation Instructions](install-gke-manual.md).
-
-Installing into GKE allows Turbinia Workers to scale based on processing demand. Currently, this is done through scaling on CPU utilization, which is determined when available Turbinia Workers process Tasks and reach a pre-defined CPU threshold. The GKE architecture closely resembles the [cloud architecture](how-it-works.md) with GKE being used to scale Turbinia Woker pods.
-
-All steps in this document are required for getting Turbinia running on GKE. 
-### **Prerequisites**
-GKE is only supported for Google Cloud so a Google Cloud Project is required to work from.
-
-## **Installation**
-
-Please follow these steps for deploying Turbinia to GKE. Ensure that the `.clusterconfig` config file has been updated appropriately. 
-
-### **Turbinia GKE Deployment**
-
-**Follow these steps to deploy Turbinia to GKE.**
-
-*   Create or select a Google Cloud Platform project in the
-    [Google Cloud Console](https://console.cloud.google.com).
-*   Determine which GCP zone and region that you wish to deploy Turbinia into.
-    Note that one of the GCP dependencies is Cloud Functions, and that only
-    works in certain regions, so you will need to deploy in one of
-    [the supported regions](https://cloud.google.com/functions/docs/locations).
-* Review the `.clusterconfig` config file and please update any of the default values if necessary based on requirements.
-* Deploy Turbinia through the following command
-    * `./k8s/tools/deploy-pubsub-gke.sh`
-* The deployment script will automatically enable GCP APIs, create the cluster and GCP resources then deploy Turbinia to the cluster. At the end of the run, you should have a fully functioning Turbinia environment within GKE to use. 
-* Run the following command if you'd like to cleanup the newly created Turbinia environment
-    * `./k8s/tools/cleanup-pubsub-gke.sh`
-    * **Note this will delete the Turbinia cluster including all processed output and log files as well as associated GCP resources** 
-
-### **Making processing requests in GKE**
-* You can either make requests via setting up a local `turbiniactl` client or through connecting to the server through the following steps.
-* Connect to cluster through `gcloud container clusters get-credentials <CLUSTER_NAME> --zone <ZONE> --project <PROJECT_NAME>`.
-* Use `kubectl get pods` to get a list of running pods.
-* Identify the pod named `turbinia-server-*` and exec into it via `kubectl exec --stdin --tty [CONTAINER-NAME] -- bash`
-* Use `turbiniactl` to kick off a request to process evidence.
-
-## **Monitoring Installation**
-Turbinia GKE has the capability to be monitored through Prometheus and Grafana. Please follow the steps outlined under the Monitoring Installation section [here](install-gke-manual.md).
diff --git a/docs/user/install-manual.md b/docs/user/install-manual.md
deleted file mode 100644
index 2865509..0000000
--- a/docs/user/install-manual.md
+++ /dev/null
@@ -1,231 +0,0 @@
-# Turbinia Local and Manual Installation Instructions
-
-## **Introduction**
-
-Turbinia can be run on the [Google Cloud Platform](https://cloud.google.com), on
-local machines, or in a hybrid mode. See the
-"[how it works](how-it-works.md)"
-documentation for more details on what the architecture looks like for each of
-these installation types. This page covers the local installation as well as the
-manual steps for hybrid and cloud installation. **If you are setting up a GCP
-cloud or hybrid installation, it is highly recommended and much simpler to use
-the
-[terraform installation method](install.md)
-to bootstrap those environments.**
-
-Each section of this document indicates which installation types it applies to
-(cloud, hybrid or local), so you only need to follow the relevant steps for your
-installation type.
-
-### **Prerequisites**
-
-Turbinia requires all worker nodes to have equal access to all Evidence data.
-For Google Cloud this means using Google Cloud Storage (GCS), but for local and
-hybrid configurations the easiest setup is to have NFS or a SAN mounted on a
-common path on each worker. All output should also be written to this common
-directory so that when new Evidence is generated by a Task, that the other
-worker nodes can access it for processing. Turbinia can also write output to GCS
-even when running locally (see the `GCS_OUTPUT_PATH` variable in the config).
-
-## **Installation**
-
-To run Turbinia it's recommended that you have at least two machines or cloud
-instances, one for the server and one or more for workers. In a small or
-development setup, you can also run both the server and worker on a single
-instance.
-
-### **GCP Setup**
-
-**This section is required for cloud and hybrid configurations.**
-
-*   Create or select a Google Cloud Platform project in the
-    [Google Cloud Console](https://console.cloud.google.com).
-*   Determine which GCP zone and region that you wish to deploy Turbinia into.
-    Note that one of the GCP dependencies is Cloud Functions, and that only
-    works in certain regions, so you will need to deploy in one of
-    [the supported regions](https://cloud.google.com/functions/docs/locations).
-*   Enable
-    [Cloud Functions](https://console.cloud.google.com/apis/library/cloudfunctions.googleapis.com).
-*   Follow the
-    [instructions](https://cloud.google.com/pubsub/docs/quickstart-console) to:
-    *   Enable
-        [Cloud Pub/Sub](https://console.cloud.google.com/apis/library/pubsub.googleapis.com)
-    *   Create a new Pub/Sub topic and subscription (pull type with 600s
-        timeout). These can use the same base name (the part after `topics/` and
-        `subscription/` in the paths).
-    *   Please take a note of the topic name for the configuration steps, as
-        this is what you will set the `PUBSUB_TOPIC` config variable to.
-*   Enable
-    [Cloud Datastore](https://console.cloud.google.com/apis/api/datastore.googleapis.com)
-    *   Go to Datastore in the cloud console
-    *   Hit the `Create Entity` button
-    *   Select the same region that you selected in the previous steps. No need
-        to create any Entities after selecting your region
-
-#### **Create a GCE Instance for the Server**
-
-**This section is required for cloud configurations.**
-
-*   Create a
-    [new GCE instance](https://console.cloud.google.com/compute/instances) from
-    a recent version of Debian or Ubuntu (currently 18.0.4 is recommended). Once
-    the host is configured we'll later clone the disk to use for the workers.
-    *   This should work on other Linux flavors, but these are untested, and the
-        installation steps would need to be adapted.
-
-#### **Create a Google Cloud Storage (GCS) Bucket**
-
-**This section is required for cloud installations.**
-
-*   [Create a new GCS bucket](https://cloud.google.com/storage/docs/creating-buckets)
-    and take note of the bucket name as this will be referenced later by the
-    `GCS_OUTPUT_PATH` variable.
-
-### **Core Installation**
-
-**This section is required for cloud, hybrid and local configurations.**
-
-#### **Preparation**
-
-All steps listed below must be completed on all servers/workers unless otherwise
-noted. If you are installing in a cloud environment, you will only have a server
-VM at this point, and we will later clone this instance to create the workers.
-
-*   Install dependencies
-    *   `sudo apt-get install python-dev build-essential python-setuptools
-        python-pip python-virtualenv liblzma-dev git john`
-*   Create a turbinia user with password-less sudo access
-    *   `sudo adduser --disabled-password turbinia`
-    *   `echo "turbinia ALL = (root) NOPASSWD:
-         /bin/mount,/bin/umount,/sbin/losetup" | sudo tee -a /etc/sudoers.d/turbinia`
-*   Prepare configuration directory:
-    *   `sudo mkdir /etc/turbinia`
-    *   `sudo chown turbinia /etc/turbinia`
-*   Checkout git at release branch for config and other setup
-    *   `git clone https://github.com/google/turbinia.git`
-    *   `cd turbinia`
-    *   `git branch -l | grep release`
-    *   `git checkout <latest release branch from previous step>`
-
-#### **Installation and Configuration**
-
-*   Log in as turbinia
-    *   `sudo su - turbinia`
-*   Install Turbinia
-    *   Note: You may want to install this into a virtual-environment with venv
-        or pipenv to reduce potential dependency conflicts and isolate these
-        packages into their own environment.
-    *   `pip3 install turbinia` for the server
-    *   `pip3 install turbinia[worker]` for the worker
-    *   `pip3 install turbinia[dev]` if you want to run tests or get the
-        development dependencies.
-    *   If you are running a local installation:
-        *   `pip3 install turbinia[local]`
-*   Install Worker binary dependencies
-    *   You can install Plaso from the
-        [GIFT PPA](https://launchpad.net/~gift/+archive/ubuntu/stable), or
-        [see here](https://plaso.readthedocs.io/en/latest/sources/user/Users-Guide.html#user-s-guide)
-        for other packaged installations.
-    *   There are a few other binary
-        dependencies that are not packaged with Ubuntu or PyPi, so these
-        will need to be installed manually:
-        [bulk_extractor](https://github.com/simsong/bulk_extractor/wiki/Installing-bulk_extractor),
-        [hindsight](https://github.com/obsidianforensics/hindsight) (this one
-        is technically in PyPi, but since it's not Python3 yet it needs to be
-        installed separately) and
-        [Volatility](https://github.com/volatilityfoundation/volatility/wiki/Installation).
-        Alternately you can disable the Jobs that have those dependencies.
-        You can do this by setting the `DISABLED_JOBS` in the
-        `/etc/turbinia/turbinia.conf` config after it is installed (see below).
-        You can see the list of Jobs with `turbiniactl listjobs` after everything
-        is set up.
-*   Create and configure the Turbinia configuration file.
-    *   `cp <git clone path>/turbinia/config/turbinia_config_tmpl.py
-        /etc/turbinia/turbinia.conf`
-    *   Alternately, you can either put the file in `/home/$USER/.turbiniarc` or
-        in another directory and then point the `TURBINIA_CONFIG_PATH`
-        environment variable to that directory.
-    *   Edit the config file to match your local installation details.
-        *   Match the `PUBSUB_TOPIC` variable in the configuration to the name
-            of the topic you created in GCP.
-        *   If you are running Turbinia locally, make sure to set
-            `GCS_OUTPUT_PATH` to `None`.
-        *   For local and hybrid installations:
-            *   Set `SHARED_FILESYSTEM = True`
-        *   For local installations:
-            *   Set `STATE_MANAGER = 'Redis'`
-            *   Set `TASK_MANAGER = 'Celery'`
-            *   Configure the `CELERY*`, `KOMBU*` and `REDIS*` variables as
-                appropriate for your config.
-        *   Set the following
-    *   Configure the `OUTPUT_DIR`, `TMP_DIR`, and `MOUNT_DIR_PREFIX` to match
-        your local system. On the worker nodes, create the corresponding
-        directories and make sure they are owned by the turbinia user.
-*   Configure the init scripts to run Turbinia on start
-    *   `cp <git clone path>/turbinia/tools/turbinia@.service
-        /etc/systemd/system/turbinia@server` for the server
-    *   `cp <git clone path>/turbinia/tools/turbinia@.service
-        /etc/systemd/system/turbinia@psqworker` for a GCP worker
-    *   `cp <git clone path>/turbinia/tools/turbinia@.service
-        /etc/systemd/system/turbinia@celeryworker` for a local (non-cloud)
-        installation.
-    *   Follow the instructions at the top of the
-        `turbinia/tools/turbinia@.service` file to enable these services.
-
-### **GCP Installation**
-
-**This section is required for cloud and hybrid configurations.**
-
-*   Install
-    [google-cloud-sdk](https://cloud.google.com/sdk/docs/quickstart-linux)
-*   Create a
-    [scoped service account](https://cloud.google.com/compute/docs/access/service-accounts)
-    (this is the best option) with the following roles:
-    *   `Cloud Datastore User`: Used by PSQ to store result data, and in the
-        future by the Task Manager to store queryable task data
-    *   `Pub/Sub Editor`: Used by clients to talk to Turbinia, and by the Task
-        Manager to talk to workers
-    *   `Storage Object Admin` and `Storage Legacy Bucket Reader`: Only required
-        on the GCS bucket used by Turbinia, if any. See
-        [GCP Turbinia](install.md)
-        for details
-    *   `Compute Instance Admin`: Used to list instances and to attach disks to
-        instances
-    *   `Service Account User`: Used when attaching disks
-    *   `Cloud Functions Developer`: Used by turbiniactl to query task status
-*   Create a new key for your service account, and save it on server/workers and
-    then configure init scripts in `/etc/systemd/system/turbinia*` to point to
-    it by setting the `GOOGLE_APPLICATION_CREDENTIALS` var similar to
-    `ExecStartPre=+/bin/sh -c '/bin/systemctl set-environment
-    GOOGLE_APPLICATION_CREDENTIALS="/home/turbinia/turbinia-service-account-creds.json"'`
-*   Add the service account to the gcloud auth
-    *   `gcloud auth list`
-    *   `gcloud auth activate-service-account
-        --key-file=$GOOGLE_APPLICATION_CREDENTIALS`
-*   Alternately you can run Turbinia under your own credentials (NOT RECOMMENDED
-    except for development environments)
-    *   Run `gcloud auth login` (may require you to copy/paste url to browser)
-    *   Run `gcloud auth application-default login`
-
-#### **Create GCP workers**
-
-**This section is required for cloud configurations.**
-
-*   Stop the server instance that has been configured above.
-*   Create a new image from the server VM's disk.
-*   Create a new Instance Template using the newly created image.
-*   Create a new Managed Instance Group from the newly created Instance
-    Template.
-
-#### **Deploy Cloud Functions**
-
-**This section is required for cloud and hybrid configurations.**
-
-*   `cd <git clone path>/tools/gcf_init && ./deploy_gcf.py`
-
-## **Local Turbinia**
-
-Running Turbinia locally using Docker. This setup does not require Google 
-Cloud Platform.
-
-See [here](https://github.com/google/turbinia/tree/master/docker/local/) for detailed instructions.
diff --git a/docs/user/install.md b/docs/user/install.md
index b1d8f0b..b11170a 100644
--- a/docs/user/install.md
+++ b/docs/user/install.md
@@ -1,6 +1,6 @@
-**Note**: ***This installation method will be deprecated by the end of 2022. 
+**Note**: **_This installation method will be deprecated by the end of 2022.
 The current recommended method for installing Turbinia is
-[here](https://turbinia.readthedocs.io/en/latest/user/install-gke.html).***
+[here](install-gke-pubsub.md)._**
 
 # **Turbinia Quick Installation Instructions**
 
@@ -15,10 +15,7 @@ instructions for Cloud installations. This uses
 [terraform configs](https://github.com/forseti-security/osdfir-infrastructure)
 that are part of the
 [Forseti Security repository](https://github.com/forseti-security)
-to automate deployment of Turbinia into an existing GCP Project. If you want to
-install Turbinia in hybrid or local only mode, or want to install Turbinia
-manually (not recommended), see
-[here](install-manual.md)
+to automate deployment of Turbinia into an existing GCP Project.
 for details.
 
 ## Installation
@@ -29,63 +26,64 @@ to get a shell with access to your GCP resources.
 
 ### GCP Project Setup
 
-*   Create or select a Google Cloud Platform project in the
-    [Google Cloud Console](https://console.cloud.google.com).
-*   Determine which GCP zone and region that you wish to deploy Turbinia into.
-    Note that one of the GCP dependencies is Cloud Functions, and that only
-    works in certain regions, so you will need to deploy in one of
-    [the supported regions](https://cloud.google.com/functions/docs/locations).
-*   Install
-    [google-cloud-sdk](https://cloud.google.com/sdk/docs/quickstart-linux).
-    *   Note: If you are doing this from cloud shell you shouldn't need this
-        step.
-*   Run `gcloud auth login` to authenticate. This may require you to copy/paste
-    url to browser.
-*   Run `gcloud auth application-default login`
+- Create or select a Google Cloud Platform project in the
+  [Google Cloud Console](https://console.cloud.google.com).
+- Determine which GCP zone and region that you wish to deploy Turbinia into.
+  Note that one of the GCP dependencies is Cloud Functions, and that only
+  works in certain regions, so you will need to deploy in one of
+  [the supported regions](https://cloud.google.com/functions/docs/locations).
+- Install
+  [google-cloud-sdk](https://cloud.google.com/sdk/docs/quickstart-linux).
+  - Note: If you are doing this from cloud shell you shouldn't need this
+    step.
+- Run `gcloud auth login` to authenticate. This may require you to copy/paste
+  url to browser.
+- Run `gcloud auth application-default login`
 
 ### Deploy Turbinia
 
-*   Download the
-    [Terraform CLI from here](https://www.terraform.io/downloads.html).
-*   Clone the Forseti Security repository and change to the path containing the
-    configs
-    *   `git clone https://github.com/forseti-security/osdfir-infrastructure/`
-    *   `cd osdfir-infrastructure`
-*   Configuration
-    *   By default this will create one Turbinia server instance and one worker
-        instance. If you want to change the number of workers, edit the
-        `modules/turbinia/variables.tf` file and set the `turbinia_worker_count`
-        variable to the number of workers you want to deploy.
-    *   To adjust the GCP zone and region you want to run Turbinia in, edit the
-        `modules/turbinia/variables.tf` file and change the `gcp_zone` and 
-        `gcp_region` variables as appropriate to reflect your GCP project's
-        zone and region.
-    *   If you want to use docker to run Turbinia tasks, please follow the
-        instructions [here](using-docker.md) to enable docker.
-    *   Running the following commands will leave some state information under
-        the current directory, so if you wish to continue to manage the number
-        of workers via Terraform you should keep this directory for later use.
-        Alternatively, if you wish to store this information in GCS instead, you
-        can edit `main.tf` and change the `bucket` parameter to the GCS bucket
-        you wish to keep this state information in. See the
-        [Terraform documentation](https://www.terraform.io/docs/commands/index.html)
-        for more information.
-    *   The current configuration does not enable alert notifications by default.
-        Please see [here](#grafana-smtp-setup) for the instructions
-    *   If you are running multiple workers on a given host and within containers, ensure
-        that you are mapping the host `OUTPUT_DIR` path specified in the configuration file 
-        `.turbiniarc` to the containers so that they can properly update the `RESOURCE_STATE_FILE`.
-
-*   Initialize terraform and apply the configuration
-    *   `./deploy.sh --no-timesketch`
-        *   If the `--no-timesketch` parameter is not supplied, Terraform will also
-            create a [Timesketch](http://timesketch.org) instance in the same
-            project, and this can be configured to ingest Turbinia timeline
-            output and report data. See the
-            [Documentation on this](https://github.com/forseti-security/osdfir-infrastructure)
-            for more details.
-        *   When prompted for the project name, enter the project you selected
-            during setup.
+- Download the
+  [Terraform CLI from here](https://www.terraform.io/downloads.html).
+- Clone the Forseti Security repository and change to the path containing the
+  configs
+  - `git clone https://github.com/forseti-security/osdfir-infrastructure/`
+  - `cd osdfir-infrastructure`
+- Configuration
+
+  - By default this will create one Turbinia server instance and one worker
+    instance. If you want to change the number of workers, edit the
+    `modules/turbinia/variables.tf` file and set the `turbinia_worker_count`
+    variable to the number of workers you want to deploy.
+  - To adjust the GCP zone and region you want to run Turbinia in, edit the
+    `modules/turbinia/variables.tf` file and change the `gcp_zone` and
+    `gcp_region` variables as appropriate to reflect your GCP project's
+    zone and region.
+  - If you want to use docker to run Turbinia tasks, please follow the
+    instructions [here](using-docker.md) to enable docker.
+  - Running the following commands will leave some state information under
+    the current directory, so if you wish to continue to manage the number
+    of workers via Terraform you should keep this directory for later use.
+    Alternatively, if you wish to store this information in GCS instead, you
+    can edit `main.tf` and change the `bucket` parameter to the GCS bucket
+    you wish to keep this state information in. See the
+    [Terraform documentation](https://www.terraform.io/docs/commands/index.html)
+    for more information.
+  - The current configuration does not enable alert notifications by default.
+    Please see [here](#grafana-smtp-setup) for the instructions
+  - If you are running multiple workers on a given host and within containers, ensure
+    that you are mapping the host `OUTPUT_DIR` path specified in the configuration file
+    `.turbiniarc` to the containers so that they can properly update the `RESOURCE_STATE_FILE`.
+
+- Initialize terraform and apply the configuration
+  - `./deploy.sh --no-timesketch`
+    - If the `--no-timesketch` parameter is not supplied, Terraform will also
+      create a [Timesketch](http://timesketch.org) instance in the same
+      project, and this can be configured to ingest Turbinia timeline
+      output and report data. See the
+      [Documentation on this](https://github.com/forseti-security/osdfir-infrastructure)
+      for more details.
+    - When prompted for the project name, enter the project you selected
+      during setup.
 
 This should result in the appropriate cloud services being enabled and
 configured and GCE instances for the server and the worker(s) being started and
@@ -95,8 +93,8 @@ workers, you can edit the `turbinia/variables.tf` file mentioned above and
 re-run `terraform apply`
 To use Turbinia you can use the virtual environment that was setup by
 the `deploy.sh` script.To activate the virtual environment, run the following
-command  `source ~/turbinia/bin/activate` and then use `turbiniactl`. For more
-information on how to use Turbinia please visit [the user manual](https://github.com/google/turbinia). 
+command `source ~/turbinia/bin/activate` and then use `turbiniactl`. For more
+information on how to use Turbinia please visit [the user manual](https://github.com/google/turbinia).
 
 ### Client configuration (optional)
 
@@ -107,31 +105,31 @@ requests. If you'd prefer to use turbiniactl on a different machine, follow the
 following instructions to configure the client. The instructions are based on
 using Ubuntu 18.04, though other versions of Linux should be compatible.
 
-*   Follow the steps from GCP Project setup above to install the SDK and
-    authenticate with gcloud.
-*   Install some python tooling:
-    *   `apt-get install python3-pip python3-wheel`
-*   Install the Turbinia client.
-    *   Note: You may want to install this into a virtual-environment with
-        [venv](https://docs.python.org/3.7/library/venv.html) or
-        [pipenv](https://pipenv.pypa.io/en/latest/)) to reduce potential
-        dependency conflicts and isolate these packages into their own
-        environment.
-    *   `pip3 --user install turbinia`
-*   If running on the same machine you deployed Turbinia from, you can generate
-    the config with terraform
-    *   `terraform output turbinia-config > ~/.turbiniarc`
-*   Otherwise, if you are running from a different machine you'll need to copy
-    the Turbinia config from the original machine, or from the Turbinia server
-    from `/etc/turbinia/turbinia.conf`.
-
-### Grafana SMTP Setup 
+- Follow the steps from GCP Project setup above to install the SDK and
+  authenticate with gcloud.
+- Install some python tooling:
+  - `apt-get install python3-pip python3-wheel`
+- Install the Turbinia client.
+  - Note: You may want to install this into a virtual-environment with
+    [venv](https://docs.python.org/3.7/library/venv.html) or
+    [pipenv](https://pipenv.pypa.io/en/latest/)) to reduce potential
+    dependency conflicts and isolate these packages into their own
+    environment.
+  - `pip3 --user install turbinia`
+- If running on the same machine you deployed Turbinia from, you can generate
+  the config with terraform
+  - `terraform output turbinia-config > ~/.turbiniarc`
+- Otherwise, if you are running from a different machine you'll need to copy
+  the Turbinia config from the original machine, or from the Turbinia server
+  from `/etc/turbinia/turbinia.conf`.
+
+### Grafana SMTP Setup
 
 If you want to receive alert notifications from Grafana, you'll need to setup a SMTP server for Grafana. To configure a SMTP server, you need to add the following environment variables to `Grafana` `env` section in `osdfir-infrastructure/modules/monitoring/main.tf`
 
 ```
       {
-        name = "GF_SMTP_ENABLED" 
+        name = "GF_SMTP_ENABLED"
         value = "true"
       }, {
         name = "GF_SMTP_HOST"
@@ -153,6 +151,7 @@ If you want to receive alert notifications from Grafana, you'll need to setup a
 ```
 
 ---
+
 > **NOTE**
 
 > By default Gmail does not allow [less secure apps](https://support.google.com/accounts/answer/6010255) to authenticate and send emails. For that reason, you'll need to allow less secure apps to access the provided Gmail account.
@@ -160,8 +159,9 @@ If you want to receive alert notifications from Grafana, you'll need to setup a
 ---
 
 Once completed:
- - login to the Grafana Dashboard.
- - Select Alerting and choose "Notification channels".
- - Fill the required fields and add the email addresses that will receive notification.
- - Click "Test" to test your SMTP setup.
- - Once everything is working, click "Save" to save the notification channel.
+
+- login to the Grafana Dashboard.
+- Select Alerting and choose "Notification channels".
+- Fill the required fields and add the email addresses that will receive notification.
+- Click "Test" to test your SMTP setup.
+- Once everything is working, click "Save" to save the notification channel.
diff --git a/k8s/celery/destroy-celery.sh b/k8s/celery/destroy-celery.sh
new file mode 100755
index 0000000..133a580
--- /dev/null
+++ b/k8s/celery/destroy-celery.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+# Turbinia Kubernetes destroy script for Celery/Redis config.
+# This script can be used to destroy the Turbinia deployment in Kubernetes.
+# Please use the destroy-celery-gke.sh script if you'd like to also delete
+# the cluster and other GCP resources created as part of the deployment.
+
+kubectl delete configmap turbinia-config
+kubectl delete -f redis-server.yaml
+kubectl delete -f redis-service.yaml
+kubectl delete -f turbinia-autoscale-cpu.yaml 
+kubectl delete -f turbinia-server-metrics-service.yaml 
+kubectl delete -f turbinia-worker-metrics-service.yaml 
+kubectl delete -f turbinia-worker.yaml 
+kubectl delete -f turbinia-server.yaml
+kubectl delete -f turbinia-volume-claim-filestore.yaml
+kubectl delete -f turbinia-volume-filestore.yaml
\ No newline at end of file
diff --git a/k8s/celery/redis-server.yaml b/k8s/celery/redis-server.yaml
new file mode 100644
index 0000000..c99a7b2
--- /dev/null
+++ b/k8s/celery/redis-server.yaml
@@ -0,0 +1,48 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: redis-server
+  labels:
+    app: redis-server
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: redis-server
+  template:
+    metadata:
+      labels:
+        app: redis-server
+    spec:
+      serviceAccountName: turbinia
+      initContainers:
+        - name: init-filestore
+          image: busybox:1.28
+          command: ["sh", "-c", "chmod og+w /mnt/turbiniavolume && echo never > /host-sys/kernel/mm/transparent_hugepage/enabled"]
+          volumeMounts:
+            - mountPath: /mnt/turbiniavolume
+              name: turbiniavolume
+            - mountPath: /host-sys
+              name: host-sys
+      containers:
+        - name: redis
+          image: "docker.io/redis:latest"
+          args: ["--appendonly", "yes", "--save", "30", "1", "--client-output-buffer-limit", "pubsub", "268435456", "67108864", "0"]
+          workingDir: /mnt/turbiniavolume/redis
+          resources:
+            requests:
+              cpu: 2000m
+              memory: 2000Mi
+          ports:
+            - containerPort: 6379
+          volumeMounts:
+            - mountPath: /mnt/turbiniavolume
+              name: turbiniavolume
+      volumes:
+        - name: turbiniavolume
+          persistentVolumeClaim:
+            claimName: turbiniavolume-claim
+            readOnly: false
+        - name: host-sys
+          hostPath:
+            path: /sys
\ No newline at end of file
diff --git a/k8s/celery/redis-service.yaml b/k8s/celery/redis-service.yaml
new file mode 100644
index 0000000..c5e46b3
--- /dev/null
+++ b/k8s/celery/redis-service.yaml
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: redis
+  labels:
+    app: redis-server
+spec:
+  ports:
+  - port: 6379
+    targetPort: 6379
+  selector:
+    app: redis-server
\ No newline at end of file
diff --git a/k8s/celery/setup-celery.sh b/k8s/celery/setup-celery.sh
new file mode 100755
index 0000000..eb742a3
--- /dev/null
+++ b/k8s/celery/setup-celery.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+# Turbinia Kubernetes deployment script for Celery/Redis config.
+# This script can be used to deploy Turbinia to a Kubernetes environment.
+# Please use the deploy-celery-gke.sh script if you'd also like to create
+# the GKE cluster and associated GCP resources required by Turbinia.
+
+TURBINIA_CONF=$1
+if [ -z $1 ]; then
+    echo "No config found as parameter, please specify a Turbinia config file."
+    exit 0
+fi
+
+base64 -w0 $TURBINIA_CONF > turbinia-config.b64
+kubectl create configmap turbinia-config --from-file=TURBINIA_CONF=turbinia-config.b64
+kubectl create -f turbinia-volume-filestore.yaml
+kubectl create -f turbinia-volume-claim-filestore.yaml
+kubectl create -f redis-server.yaml
+kubectl create -f redis-service.yaml
+kubectl rollout status -w deployment/redis-server
+kubectl create -f turbinia-server.yaml
+kubectl create -f turbinia-worker.yaml
+kubectl create -f turbinia-server-metrics-service.yaml
+kubectl create -f turbinia-worker-metrics-service.yaml
+kubectl create -f turbinia-autoscale-cpu.yaml
+
+echo "Turbinia deployment complete"
diff --git a/k8s/celery/turbinia-autoscale-cpu.yaml b/k8s/celery/turbinia-autoscale-cpu.yaml
new file mode 100644
index 0000000..51d6685
--- /dev/null
+++ b/k8s/celery/turbinia-autoscale-cpu.yaml
@@ -0,0 +1,12 @@
+apiVersion: autoscaling/v1
+kind: HorizontalPodAutoscaler
+metadata:
+  name: turbinia-worker-autoscaling
+spec:
+  scaleTargetRef:
+    apiVersion: apps/v1
+    kind: Deployment
+    name: turbinia-worker
+  minReplicas: 3
+  maxReplicas: 50
+  targetCPUUtilizationPercentage: 90
diff --git a/k8s/celery/turbinia-server-metrics-service.yaml b/k8s/celery/turbinia-server-metrics-service.yaml
new file mode 100644
index 0000000..8c58c62
--- /dev/null
+++ b/k8s/celery/turbinia-server-metrics-service.yaml
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: turbinia-server-metrics
+  labels:
+    app: turbinia-server-metrics
+spec:
+  ports:
+  - port: 9200
+    targetPort: 9200
+  selector:
+    app: turbinia-server
\ No newline at end of file
diff --git a/k8s/celery/turbinia-server.yaml b/k8s/celery/turbinia-server.yaml
new file mode 100644
index 0000000..ce1338a
--- /dev/null
+++ b/k8s/celery/turbinia-server.yaml
@@ -0,0 +1,52 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: turbinia-server
+  labels:
+    app: turbinia-server
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: turbinia-server
+  template:
+    metadata:
+      annotations:
+        prometheus.io/port: "9200"
+        prometheus.io/scrape: "true"
+      labels:
+        app: turbinia-server
+    spec:
+      serviceAccountName: turbinia
+      initContainers:
+        - name: init-filestore
+          image: busybox:1.28
+          command: ['sh', '-c', 'chmod go+w /mnt/turbiniavolume']
+          volumeMounts:
+            - mountPath: "/mnt/turbiniavolume"
+              name: turbiniavolume
+      containers:
+        - name: server
+          image: us-docker.pkg.dev/osdfir-registry/turbinia/release/turbinia-server:latest
+          env:
+            - name: TURBINIA_CONF
+              valueFrom:
+                configMapKeyRef:
+                  name: turbinia-config
+                  key: TURBINIA_CONF
+            - name: TURBINIA_EXTRA_ARGS
+              value: "-d"
+            - name: NODE_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+          volumeMounts:
+            - mountPath: /mnt/turbiniavolume
+              name: turbiniavolume
+          ports:
+            - containerPort: 9200
+      volumes:
+        - name: turbiniavolume
+          persistentVolumeClaim:
+            claimName: turbiniavolume-claim
+            readOnly: false
\ No newline at end of file
diff --git a/k8s/celery/turbinia-volume-claim-filestore.yaml b/k8s/celery/turbinia-volume-claim-filestore.yaml
new file mode 100644
index 0000000..1fd326c
--- /dev/null
+++ b/k8s/celery/turbinia-volume-claim-filestore.yaml
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: turbiniavolume-claim
+spec:
+  accessModes:
+  - ReadWriteMany
+  storageClassName: ""
+  volumeName: turbiniavolume
+  resources:
+    requests:
+      storage: 1T
\ No newline at end of file
diff --git a/k8s/celery/turbinia-volume-filestore.yaml b/k8s/celery/turbinia-volume-filestore.yaml
new file mode 100644
index 0000000..1b95ef1
--- /dev/null
+++ b/k8s/celery/turbinia-volume-filestore.yaml
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: turbiniavolume
+spec:
+  capacity:
+    storage: 1T
+  accessModes:
+  - ReadWriteMany
+  nfs:
+    path: /turbiniavolume
+    server: <IP_ADDRESS>
\ No newline at end of file
diff --git a/k8s/celery/turbinia-worker-metrics-service.yaml b/k8s/celery/turbinia-worker-metrics-service.yaml
new file mode 100644
index 0000000..6dd22ad
--- /dev/null
+++ b/k8s/celery/turbinia-worker-metrics-service.yaml
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: turbinia-worker-metrics
+  labels:
+    app: turbinia-worker-metrics
+spec:
+  ports:
+  - port: 9200
+    targetPort: 9200
+  selector:
+    app: turbinia-worker
\ No newline at end of file
diff --git a/k8s/celery/turbinia-worker.yaml b/k8s/celery/turbinia-worker.yaml
new file mode 100644
index 0000000..4a66768
--- /dev/null
+++ b/k8s/celery/turbinia-worker.yaml
@@ -0,0 +1,83 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: turbinia-worker
+  labels:
+    app: turbinia-worker
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      app: turbinia-worker
+  template:
+    metadata:
+      annotations:
+        prometheus.io/port: "9200"
+        prometheus.io/scrape: "true"
+      labels:
+        app: turbinia-worker
+    spec:
+      serviceAccountName: turbinia
+      # The grace period needs to be set to the largest task timeout as
+      # set in the turbinia configuration file.
+      terminationGracePeriodSeconds: 86400
+      initContainers:
+        - name: init-filestore
+          image: busybox:1.28
+          command: ['sh', '-c', 'chmod go+w /mnt/turbiniavolume']
+          volumeMounts:
+            - mountPath: "/mnt/turbiniavolume"
+              name: turbiniavolume
+      containers:
+        - name: worker
+          image: us-docker.pkg.dev/osdfir-registry/turbinia/release/turbinia-worker:latest
+          lifecycle:
+            preStop:
+              exec:
+                command:
+                  - "/bin/sh"
+                  - "-c"
+                  - "touch /tmp/turbinia-to-scaledown.lock && sleep 5 && /usr/bin/python3 /home/turbinia/check-lockfile.py"
+          securityContext:
+            privileged: true
+          env:
+            - name: TURBINIA_CONF
+              valueFrom:
+                configMapKeyRef:
+                  name: turbinia-config
+                  key: TURBINIA_CONF
+            - name: TURBINIA_EXTRA_ARGS
+              value: "-d"
+            - name: NODE_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+          volumeMounts:
+            - mountPath: "/dev"
+              name: dev
+              readOnly: true
+            - mountPath: "/var/run/lock"
+              name: lockfolder
+              readOnly: false
+            - mountPath: /mnt/turbiniavolume
+              name: turbiniavolume
+          ports:
+            - containerPort: 9200
+          resources: 
+            requests:
+              memory: "256Mi"
+              cpu: "500m"
+            limits:
+              memory: "8192Mi"
+              cpu: "32000m"
+      volumes:
+        - name: dev
+          hostPath:
+            path: /dev
+        - name: lockfolder
+          hostPath:
+            path: /var/run/lock
+        - name: turbiniavolume
+          persistentVolumeClaim:
+            claimName: turbiniavolume-claim
+            readOnly: false
diff --git a/k8s/tools/deploy-celery-gke.sh b/k8s/tools/deploy-celery-gke.sh
new file mode 100755
index 0000000..a920bbe
--- /dev/null
+++ b/k8s/tools/deploy-celery-gke.sh
@@ -0,0 +1,242 @@
+#!/bin/bash
+# Turbinia GKE deployment script.
+# This script can be used to deploy the Turbinia Celery stack to GKE. 
+# Requirements:
+# - have 'gcloud' and 'kubectl' installed.
+# - autheticate against your GCP project with "gcloud auth login"
+# - account being used to run script should have an IAM policy of instance.admin and container.admin used to create the necessary resources.
+# - optionally have the GCP project set with "gcloud config set project [you-project-name]"
+#
+# Use --help to show you commands supported.
+
+set -o posix
+set -e
+
+# Source cluster config to pull specs to create cluster from. Please review
+# the config file and make any necessary changes prior to executing this script
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+source $DIR/.clusterconfig
+cd $DIR/..
+
+if [[ "$*" == *--help ||  "$*" == *-h ]] ; then
+  echo "Turbinia deployment script for Kubernetes environment"
+  echo "Options:"
+  echo "--build-dev                    Deploy Turbinia development docker image"
+  echo "--build-experimental           Deploy Turbinia experimental docker image"
+  echo "--no-cluster                   Do not create the cluster"
+  echo "--no-filestore                 Do not deploy Turbinia Filestore"
+  echo "--deploy-dfdewey               Deploy dfDewey datastores"
+  exit 1
+fi
+
+# Check if gcloud is installed
+if [[ -z "$( which gcloud )" ]] ; then
+  echo "gcloud CLI not found.  Please follow the instructions at "
+  echo "https://cloud.google.com/sdk/docs/install to install the gcloud "
+  echo "package first."
+  exit 1
+fi
+
+# Check if kubectl is installed
+if [[ -z "$( which kubectl )" ]] ; then
+  echo "kubectl CLI not found.  Please follow the instructions at "
+  echo "https://kubernetes.io/docs/tasks/tools/ to install the kubectl "
+  echo "package first."
+  exit 1
+fi
+
+# Check configured gcloud project
+if [[ -z "$DEVSHELL_PROJECT_ID" ]] ; then
+  DEVSHELL_PROJECT_ID=$(gcloud config get-value project)
+  ERRMSG="ERROR: Could not get configured project. Please either restart "
+  ERRMSG+="Google Cloudshell, or set configured project with "
+  ERRMSG+="'gcloud config set project PROJECT' when running outside of Cloudshell."
+  if [[ -z "$DEVSHELL_PROJECT_ID" ]] ; then
+    echo $ERRMSG
+    exit 1
+  fi
+  echo "Environment variable \$DEVSHELL_PROJECT_ID was not set at start time "
+  echo "so attempting to get project config from gcloud config."
+  echo -n "Do you want to use $DEVSHELL_PROJECT_ID as the target project? (y / n) > "
+  read response
+  if [[ $response != "y" && $response != "Y" ]] ; then
+    echo $ERRMSG
+    exit 1
+  fi
+fi
+
+# TODO: Do real check to make sure credentials have adequate roles
+if [[ $( gcloud -q --project $DEVSHELL_PROJECT_ID auth list --filter="status:ACTIVE" --format="value(account)" | wc -l ) -eq 0 ]] ; then
+  echo "No gcloud credentials found.  Use 'gcloud auth login' and 'gcloud auth application-default login' to log in"
+  exit 1
+fi
+
+# Enable IAM services
+gcloud -q --project $DEVSHELL_PROJECT_ID services enable iam.googleapis.com
+
+# Create Turbinia service account with necessary IAM roles. The service account will be used at
+# container runtime in order to have the necessary permissions to attach and detach GCP disks as
+# well as write to stackdriver logging and error reporting.
+SA_NAME="turbinia"
+SA_MEMBER="serviceAccount:$SA_NAME@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com"
+if [[ -z "$(gcloud -q --project $DEVSHELL_PROJECT_ID iam service-accounts list --format='value(name)' --filter=name:/$SA_NAME@)" ]] ; then
+  gcloud --project $DEVSHELL_PROJECT_ID iam service-accounts create "${SA_NAME}" --display-name "${SA_NAME}"
+  # Grant IAM roles to the service account
+  echo "Grant permissions on service account"
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/compute.instanceAdmin'
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/logging.logWriter'
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/errorreporting.writer'
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/iam.serviceAccountUser'
+fi
+
+echo "Enabling Compute API"
+gcloud -q --project $DEVSHELL_PROJECT_ID services enable compute.googleapis.com
+
+# Check if the configured VPC network exists.
+networks=$(gcloud -q --project $DEVSHELL_PROJECT_ID compute networks list --filter="name=$VPC_NETWORK" |wc -l)
+if [[ "${networks}" -lt "2" ]]; then
+  echo "ERROR: VPC network $VPC_NETWORK not found, please create this first."
+  exit 1
+fi
+
+# Update Docker image if flag was provided else use default
+if [[ "$*" == *--build-dev* ]] ; then
+  TURBINIA_SERVER_IMAGE="us-docker.pkg.dev\/osdfir-registry\/turbinia\/release\/turbinia-server-dev:latest"
+  TURBINIA_WORKER_IMAGE="us-docker.pkg.dev\/osdfir-registry\/turbinia\/release\/turbinia-worker-dev:latest"
+elif [[ "$*" == *--build-experimental* ]] ; then
+  TURBINIA_SERVER_IMAGE="us-docker.pkg.dev\/osdfir-registry\/turbinia\/release\/turbinia-server-experimental:latest"
+  TURBINIA_WORKER_IMAGE="us-docker.pkg.dev\/osdfir-registry\/turbinia\/release\/turbinia-worker-experimental:latest"
+fi
+
+echo "Setting docker image to $TURBINIA_SERVER_IMAGE and $TURBINIA_WORKER_IMAGE"
+echo "Deploying cluster to project $DEVSHELL_PROJECT_ID"
+
+# Setup appropriate directories and copy of deployment templates and Turbinia config
+echo "Copying over template deployment files to $DEPLOYMENT_FOLDER"
+mkdir -p $DEPLOYMENT_FOLDER
+cp celery/* $DEPLOYMENT_FOLDER
+if [[ "$*" == *--deploy-dfdewey* ]] ; then
+  cp dfdewey/* $DEPLOYMENT_FOLDER
+fi
+cp ../turbinia/config/turbinia_config_tmpl.py $DEPLOYMENT_FOLDER/$TURBINIA_CONFIG
+
+# Create GKE cluster and authenticate to it
+if [[ "$*" != *--no-cluster* ]] ; then
+  echo "Enabling Container API"
+  gcloud -q --project $DEVSHELL_PROJECT_ID services enable container.googleapis.com
+  echo "Creating cluser $CLUSTER_NAME with $CLUSTER_NODE_SIZE node(s) configured with machine type $CLUSTER_MACHINE_TYPE and disk size $CLUSTER_MACHINE_SIZE"
+  gcloud -q --project $DEVSHELL_PROJECT_ID container clusters create $CLUSTER_NAME --machine-type $CLUSTER_MACHINE_TYPE --disk-size $CLUSTER_MACHINE_SIZE --num-nodes $CLUSTER_NODE_SIZE --master-ipv4-cidr $VPC_CONTROL_PANE --network $VPC_NETWORK --zone $ZONE --shielded-secure-boot --shielded-integrity-monitoring --no-enable-master-authorized-networks --enable-private-nodes --enable-ip-alias --scopes "https://www.googleapis.com/auth/cloud-platform" --labels "turbinia-infra=true" --workload-pool=$DEVSHELL_PROJECT_ID.svc.id.goog
+else
+  echo "--no-cluster specified. Authenticating to pre-existing cluster $CLUSTER_NAME"
+fi
+
+# Authenticate to cluster
+gcloud -q --project $DEVSHELL_PROJECT_ID container clusters get-credentials $CLUSTER_NAME --zone $ZONE
+# Create Kubernetes service account
+kubectl get serviceaccounts $SA_NAME || kubectl create serviceaccount $SA_NAME --namespace default
+gcloud iam service-accounts add-iam-policy-binding $SA_NAME@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com --role roles/iam.workloadIdentityUser --member "serviceAccount:$DEVSHELL_PROJECT_ID.svc.id.goog[default/$SA_NAME]"
+kubectl annotate serviceaccount $SA_NAME --overwrite --namespace default iam.gke.io/gcp-service-account=$SA_NAME@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com
+
+# Go to deployment folder to make changes files
+cd $DEPLOYMENT_FOLDER
+
+# Add service account to deployments
+sed -i -e "s/serviceAccountName: .*/serviceAccountName: $SA_NAME/g" turbinia-server.yaml turbinia-worker.yaml redis-server.yaml
+
+# Update Turbinia config with project info
+echo "Updating $TURBINIA_CONFIG config with project info"
+sed -i -e "s/^INSTANCE_ID = .*$/INSTANCE_ID = '$INSTANCE_ID'/g" $TURBINIA_CONFIG
+sed -i -e "s/^TURBINIA_PROJECT = .*$/TURBINIA_PROJECT = '$DEVSHELL_PROJECT_ID'/g" $TURBINIA_CONFIG
+sed -i -e "s/^TURBINIA_ZONE = .*$/TURBINIA_ZONE = '$ZONE'/g" $TURBINIA_CONFIG
+sed -i -e "s/^TURBINIA_REGION = .*$/TURBINIA_REGION = '$REGION'/g" $TURBINIA_CONFIG
+sed -i -e "s/^CLOUD_PROVIDER = .*$/CLOUD_PROVIDER = 'GCP'/g" $TURBINIA_CONFIG
+
+# Create File Store instance and update deployment files with created instance
+if [[ "$*" != *--no-filestore* ]] ; then  
+  echo "Enabling GCP Filestore API"
+  gcloud -q --project $DEVSHELL_PROJECT_ID services enable file.googleapis.com
+  echo "Creating Filestore instance $FILESTORE_NAME with capacity $FILESTORE_CAPACITY"
+  gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances create $FILESTORE_NAME --file-share=name=$FILESTORE_NAME,capacity=$FILESTORE_CAPACITY --zone=$ZONE --network=name=$VPC_NETWORK
+else
+  echo "Using pre existing Filestore instance $FILESTORE_NAME with capacity $FILESTORE_CAPACITY"
+fi
+
+echo "Updating $TURBINIA_CONFIG config with Filestore configuration and setting output directories"
+FILESTORE_IP=$(gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances describe $FILESTORE_NAME --zone=$ZONE --format='value(networks.ipAddresses)' --flatten="networks[].ipAddresses[]")
+FILESTORE_LOGS="'\/mnt\/$FILESTORE_NAME\/logs'"
+FILESTORE_OUTPUT="'\/mnt\/$FILESTORE_NAME\/output'"
+sed -i -e "s/<IP_ADDRESS>/$FILESTORE_IP/g" turbinia-volume-filestore.yaml
+sed -i -e "s/turbiniavolume/$FILESTORE_NAME/g" turbinia-volume-filestore.yaml turbinia-volume-claim-filestore.yaml turbinia-server.yaml turbinia-worker.yaml redis-server.yaml
+sed -i -e "s/storage: .*/storage: $FILESTORE_CAPACITY/g" turbinia-volume-filestore.yaml turbinia-volume-claim-filestore.yaml
+sed -i -e "s/^LOG_DIR = .*$/LOG_DIR = $FILESTORE_LOGS/g" $TURBINIA_CONFIG
+sed -i -e "s/^MOUNT_DIR_PREFIX = .*$/MOUNT_DIR_PREFIX = '\/mnt\/turbinia'/g" $TURBINIA_CONFIG
+sed -i -e "s/^SHARED_FILESYSTEM = .*$/SHARED_FILESYSTEM = True/g" $TURBINIA_CONFIG
+sed -i -e "s/^OUTPUT_DIR = .*$/OUTPUT_DIR = $FILESTORE_OUTPUT/g" $TURBINIA_CONFIG
+
+# Update Turbinia config with Redis/Celery parameters
+echo "Updating $TURBINIA_CONFIG with Redis/Celery config"
+sed -i -e "s/^TASK_MANAGER = .*$/TASK_MANAGER = 'Celery'/g" $TURBINIA_CONFIG
+sed -i -e "s/^STATE_MANAGER = .*$/STATE_MANAGER = 'Redis'/g" $TURBINIA_CONFIG
+sed -i -e "s/^REDIS_HOST = .*$/REDIS_HOST = 'redis.default.svc.cluster.local'/g" $TURBINIA_CONFIG
+sed -i -e "s/^DEBUG_TASKS = .*$/DEBUG_TASKS = True/g" $TURBINIA_CONFIG
+
+# Enable Stackdriver Logging and Stackdriver Traceback
+echo "Enabling Cloud Error Reporting and Logging APIs"
+gcloud -q --project $DEVSHELL_PROJECT_ID services enable clouderrorreporting.googleapis.com
+gcloud -q --project $DEVSHELL_PROJECT_ID services enable logging.googleapis.com
+echo "Updating $TURBINIA_CONFIG to enable Stackdriver Traceback and Logging"
+sed -i -e "s/^STACKDRIVER_LOGGING = .*$/STACKDRIVER_LOGGING = True/g" $TURBINIA_CONFIG
+sed -i -e "s/^STACKDRIVER_TRACEBACK = .*$/STACKDRIVER_TRACEBACK = True/g" $TURBINIA_CONFIG
+
+# Enable Prometheus
+echo "Updating $TURBINIA_CONFIG to enable Prometheus application metrics"
+sed -i -e "s/^PROMETHEUS_ENABLED = .*$/PROMETHEUS_ENABLED = True/g" $TURBINIA_CONFIG
+
+# Disable some jobs
+echo "Updating $TURBINIA_CONFIG with disabled jobs"
+sed -i -e "s/^DISABLED_JOBS = .*$/DISABLED_JOBS = $DISABLED_JOBS/g" $TURBINIA_CONFIG
+
+# Set appropriate docker image in deployment file if user specified
+if [[ ! -z "$TURBINIA_SERVER_IMAGE" && ! -z "$TURBINIA_WORKER_IMAGE" ]] ; then
+  echo "Updating deployment files with docker image $TURBINIA_SERVER_IMAGE and $TURBINIA_WORKER_IMAGE"
+  sed -i -e "s/us-docker.pkg.dev\/osdfir-registry\/turbinia\/release\/turbinia-server:latest$/$TURBINIA_SERVER_IMAGE/g" turbinia-server.yaml
+  sed -i -e "s/us-docker.pkg.dev\/osdfir-registry\/turbinia\/release\/turbinia-worker:latest$/$TURBINIA_WORKER_IMAGE/g" turbinia-worker.yaml
+fi
+
+# Deploy to cluster
+echo "Deploying Turbinia to $CLUSTER_NAME cluster"
+./setup-celery.sh $TURBINIA_CONFIG
+
+# Deploy dfDewey
+if [[ "$*" == *--deploy-dfdewey* ]] ; then
+  echo "Deploying dfDewey datastores to $CLUSTER_NAME cluster"
+  if [[ -z "$(gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances list --format='value(name)' --filter=name:$FILESTORE_DFDEWEY_NAME)" ]] ; then
+    echo "Creating Filestore instance $FILESTORE_DFDEWEY_NAME with capacity $FILESTORE_DFDEWEY_CAPACITY"
+    gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances create $FILESTORE_DFDEWEY_NAME --file-share=name=$FILESTORE_DFDEWEY_NAME,capacity=$FILESTORE_DFDEWEY_CAPACITY --zone=$ZONE --network=name=$VPC_NETWORK
+  else
+    echo "Using pre existing Filestore instance $FILESTORE_DFDEWEY_NAME with capacity $FILESTORE_DFDEWEY_CAPACITY"
+  fi
+  FILESTORE_DFDEWEY_IP=$(gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances describe $FILESTORE_DFDEWEY_NAME --zone=$ZONE --format='value(networks.ipAddresses)' --flatten="networks[].ipAddresses[]")
+  sed -i -e "s/<PATH>/$FILESTORE_DFDEWEY_NAME/g" dfdewey-volume-filestore.yaml
+  sed -i -e "s/<IP_ADDRESS>/$FILESTORE_DFDEWEY_IP/g" dfdewey-volume-filestore.yaml
+  sed -i -e "s/<CAPACITY>/$FILESTORE_DFDEWEY_CAPACITY/g" dfdewey-volume-filestore.yaml dfdewey-volume-claim-filestore.yaml
+  sed -i -e "s/<PATH>/$FILESTORE_PG_PATH/g" postgres-server.yaml
+  sed -i -e "s/<PATH>/$FILESTORE_OS_PATH/g" opensearch-server.yaml
+
+  ./setup-dfdewey.sh $TURBINIA_CONFIG
+fi
+
+# Create backup of turbinia config file if it exists
+TURBINIA_OUT="$HOME/.turbiniarc"
+if [[ -a $TURBINIA_OUT ]] ; then
+  backup_file="${TURBINIA_OUT}.$( date +%s )"
+  mv $TURBINIA_OUT $backup_file
+  echo "Backing up old Turbinia config $TURBINIA_CONFIG to $backup_file"
+fi
+
+# Make a copy of Turbinia config in user home directory
+echo "Creating a copy of Turbinia config in $TURBINIA_OUT"
+cp $TURBINIA_CONFIG $TURBINIA_OUT
+
+echo "Turbinia GKE was succesfully deployed!"
+echo "Authenticate via: gcloud container clusters get-credentials $CLUSTER_NAME --zone $ZONE" 
\ No newline at end of file
diff --git a/k8s/tools/destroy-celery-gke.sh b/k8s/tools/destroy-celery-gke.sh
new file mode 100755
index 0000000..07b8ed7
--- /dev/null
+++ b/k8s/tools/destroy-celery-gke.sh
@@ -0,0 +1,110 @@
+#!/bin/bash
+# Turbinia GKE cleanup script for Celery configuration.
+# This script can be used to cleanup the Turbinia Celery stack within GKE. Note that
+# this script will not disable any APIs to avoid outage with any other applications 
+# deployed within the project. 
+# Requirements:
+# - have 'gcloud'installed.
+# - autheticate against your GCP project with "gcloud auth login"
+# - account being used to run script should have an IAM policy of instance.admin and container.admin used to delete resources.
+# - optionally have the GCP project set with "gcloud config set project [you-project-name]"
+#
+# Use --help to show you commands supported.
+
+set -o posix
+set -e
+
+# Source cluster config to pull specs to create cluster from. Please review
+# the config file and ensure the parameters are set to the cluster you are 
+# intending to cleanup
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+source $DIR/.clusterconfig
+cd $DIR/..
+
+if [[ "$*" == *--help ||  "$*" == *-h ]] ; then
+  echo "Turbinia cleanup script for Turbinia within Kubernetes"
+  echo "Options:"
+  echo "--no-service-account           Do not delete the Turbinia service account"
+  echo "--no-filestore                 Do not cleanup Turbinia Filestore share"
+  echo "--no-dfdewey                   Do not cleanup dfDewey Filestore share"
+  echo "--no-cluster                   Do not delete the cluster"
+  exit 1
+fi
+
+# Before proceeding, prompt user to confirm deletion
+echo "This script is going to do a lot of destructive/irrecoverable actions such as deleting all output, logs, and GCP resources. "
+echo -n "Please enter in 'delete all' if you'd like to proceed: "
+read response
+if [[ $response != "delete all" ]] ; then
+  echo "'delete all' not specified. Exiting."
+  exit 1
+fi
+
+# Check configured gcloud project
+if [[ -z "$DEVSHELL_PROJECT_ID" ]] ; then
+  DEVSHELL_PROJECT_ID=$(gcloud config get-value project)
+  ERRMSG="ERROR: Could not get configured project. Please either restart "
+  ERRMSG+="Google Cloudshell, or set configured project with "
+  ERRMSG+="'gcloud config set project PROJECT' when running outside of Cloudshell."
+  if [[ -z "$DEVSHELL_PROJECT_ID" ]] ; then
+    echo $ERRMSG
+    exit 1
+  fi
+  echo "Environment variable \$DEVSHELL_PROJECT_ID was not set at start time "
+  echo "so attempting to get project config from gcloud config."
+  echo -n "Do you want to use $DEVSHELL_PROJECT_ID as the target project? (y / n) > "
+  read response
+  if [[ $response != "y" && $response != "Y" ]] ; then
+    echo $ERRMSG
+    exit 1
+  fi
+fi
+
+# Use either service account or local `gcloud auth` credentials.
+if [[ "$*" == *--no-gcloud-auth* ]] ; then
+  export GOOGLE_APPLICATION_CREDENTIALS=~/$INSTANCE_ID.json
+# TODO: Do real check to make sure credentials have adequate roles
+elif [[ $( gcloud -q --project $DEVSHELL_PROJECT_ID auth list --filter="status:ACTIVE" --format="value(account)" | wc -l ) -eq 0 ]] ; then
+  echo "No gcloud credentials found.  Use 'gcloud auth login' and 'gcloud auth application-default login' to log in"
+  exit 1
+fi
+
+# Delete the cluster
+if [[ "$*" != *--no-cluster* ]] ; then
+  echo "Deleting cluster $CLUSTER_NAME"
+  gcloud -q --project $DEVSHELL_PROJECT_ID container clusters delete $CLUSTER_NAME --zone $ZONE
+fi
+
+# Delete the Filestore instance
+if [[ "$*" != *--no-filestore* ]] ; then
+  echo "Deleting Filestore instance $FILESTORE_NAME"
+  gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances delete $FILESTORE_NAME --zone $ZONE
+fi
+# Delete the dfDewey Filestore instance
+if [[ "$*" != *--no-dfdewey* ]] ; then
+  if [[ -z "$(gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances list --format='value(name)' --filter=name:$FILESTORE_DFDEWEY_NAME)" ]] ; then
+    echo "Filestore instance $FILESTORE_DFDEWEY_NAME does not exist"
+  else
+    echo "Deleting Filestore instance $FILESTORE_DFDEWEY_NAME"
+    gcloud -q --project $DEVSHELL_PROJECT_ID filestore instances delete $FILESTORE_DFDEWEY_NAME --zone $ZONE
+  fi
+fi
+
+# Remove the service account if it was being used.
+if [[ "$*" != *--no-service-account* ]] ; then
+  SA_NAME="turbinia"
+  SA_MEMBER="serviceAccount:$SA_NAME@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com"
+
+  # Delete IAM roles from the service account
+  echo "Delete permissions on service account"
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/compute.instanceAdmin'
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/logging.logWriter'
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/errorreporting.writer'
+  gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member=$SA_MEMBER --role='roles/iam.serviceAccountUser'
+
+  # Delete service account
+  echo "Delete service account"
+  gcloud -q --project $DEVSHELL_PROJECT_ID iam service-accounts delete "${SA_NAME}@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com"
+fi
+
+echo "The Turbinia deployment $INSTANCE_ID was succesfully removed from $DEVSHELL_PROJECT_ID"
\ No newline at end of file
diff --git a/k8s/tools/update-gke-infra.sh b/k8s/tools/update-gke-infra.sh
index 195affd..75c4461 100755
--- a/k8s/tools/update-gke-infra.sh
+++ b/k8s/tools/update-gke-infra.sh
@@ -138,6 +138,7 @@ function update_config {
     CONFIG_BASE64=`cat $CONFIG_FILE | base64 -w 0`
     # Update ConfigMap with new Turbinia config
     $KUBECTL create configmap turbinia-config --from-literal=TURBINIA_CONF=$CONFIG_BASE64 -o yaml --dry-run=client | $KUBECTL replace -f -
+    rollout_restart
 }
 
 function show_deployment {
diff --git a/turbinia/__init__.py b/turbinia/__init__.py
index 37a748d..f90740c 100644
--- a/turbinia/__init__.py
+++ b/turbinia/__init__.py
@@ -38,7 +38,7 @@ def log_and_report(message, trace):
   log.error(trace)
   # If GCP Error Reporting is enabled.
   config.LoadConfig()
-  if config.STACKDRIVER_TRACEBACK:
+  if config.CLOUD_PROVIDER and config.STACKDRIVER_TRACEBACK:
     # Only load google_cloud if needed
     from turbinia.lib import google_cloud
     client = google_cloud.setup_stackdriver_traceback(config.TURBINIA_PROJECT)
diff --git a/turbinia/client.py b/turbinia/client.py
index 6caab5d..904a6a3 100644
--- a/turbinia/client.py
+++ b/turbinia/client.py
@@ -43,9 +43,9 @@ MAX_RETRIES = 10
 RETRY_SLEEP = 60
 
 config.LoadConfig()
-if config.TASK_MANAGER.lower() == 'psq':
+if config.CLOUD_PROVIDER:
   from libcloudforensics.providers.gcp.internal import function as gcp_function
-elif config.TASK_MANAGER.lower() == 'celery':
+if config.TASK_MANAGER.lower() == 'celery':
   from turbinia.state_manager import RedisStateManager
 
 log = logging.getLogger('turbinia')
diff --git a/turbinia/config/__init__.py b/turbinia/config/__init__.py
index d837699..9ab9295 100644
--- a/turbinia/config/__init__.py
+++ b/turbinia/config/__init__.py
@@ -44,6 +44,7 @@ CONFIG_MSG = (
 REQUIRED_VARS = [
     # Turbinia Config
     'INSTANCE_ID',
+    'CLOUD_PROVIDER',
     'STATE_MANAGER',
     'TASK_MANAGER',
     'LOG_DIR',
diff --git a/turbinia/config/logger.py b/turbinia/config/logger.py
index d100d94..16399e7 100644
--- a/turbinia/config/logger.py
+++ b/turbinia/config/logger.py
@@ -70,6 +70,9 @@ def setup(need_file_handler=True, need_stream_handler=True, log_file_path=None):
 
     # Check if a user specified log path was provided else create default path
     if not log_file_path:
+      # Create LOG directory if it doesn't exist
+      if not os.path.exists(config.LOG_DIR):
+        os.mkdir(config.LOG_DIR)
       log_name = os.uname().nodename
       # Check if NODE_NAME available for GKE setups
       if ENVNODENAME in os.environ:
diff --git a/turbinia/config/turbinia_config_tmpl.py b/turbinia/config/turbinia_config_tmpl.py
index 71acf2c..ed4f935 100644
--- a/turbinia/config/turbinia_config_tmpl.py
+++ b/turbinia/config/turbinia_config_tmpl.py
@@ -24,6 +24,10 @@
 # separate when running with the same Cloud projects or backend servers.
 INSTANCE_ID = 'turbinia-instance1'
 
+# Which Cloud provider to use. Valid options are None and 'GCP'. Use 'GCP'
+# for GCP or hybrid installations, and None for local installations.
+CLOUD_PROVIDER = 'GCP'
+
 # Which state manager to use. Valid options are 'Datastore' or 'Redis'.  Use
 # 'Datastore' for Cloud (GCP) or hybrid installations, and 'Redis' for local
 # installations.
@@ -283,11 +287,16 @@ STACKDRIVER_TRACEBACK = False
 # Options in this section are required if TASK_MANAGER is set to 'Celery'
 ################################################################################
 
+# Use Redis for state management
+REDIS_HOST = 'redis'
+REDIS_PORT = '6379'
+REDIS_DB = '0'
+
 # Method for communication between nodes
-CELERY_BROKER = 'redis://redis'
+CELERY_BROKER = 'redis://%s' % REDIS_HOST
 
 # Storage for task results/status
-CELERY_BACKEND = 'redis://redis'
+CELERY_BACKEND = 'redis://%s' % REDIS_HOST
 
 # Can be the same as CELERY_BROKER
 KOMBU_BROKER = CELERY_BROKER
@@ -299,11 +308,6 @@ KOMBU_CHANNEL = '%s-kombu' % INSTANCE_ID
 # http://docs.celeryproject.org/projects/kombu/en/4.0/reference/kombu.html#kombu.Connection.SimpleBuffer
 KOMBU_DURABLE = True
 
-# Use Redis for state management
-REDIS_HOST = 'redis'
-REDIS_PORT = '6379'
-REDIS_DB = '0'
-
 ################################################################################
 #                           Email Config
 #
diff --git a/turbinia/evidence.py b/turbinia/evidence.py
index 482da39..5b749f5 100644
--- a/turbinia/evidence.py
+++ b/turbinia/evidence.py
@@ -32,7 +32,7 @@ from turbinia.processors import mount_local
 from turbinia.processors import resource_manager
 
 config.LoadConfig()
-if config.TASK_MANAGER.lower() == 'psq':
+if config.CLOUD_PROVIDER:
   from turbinia.processors import google_cloud
 
 log = logging.getLogger('turbinia')
diff --git a/turbinia/processors/google_cloud.py b/turbinia/processors/google_cloud.py
index dd33cfe..e72d940 100644
--- a/turbinia/processors/google_cloud.py
+++ b/turbinia/processors/google_cloud.py
@@ -65,10 +65,12 @@ def GetLocalInstanceName():
   """
   # TODO(aarontp): Use cloud API instead of manual requests to metadata service.
   req = urllib.request.Request(
-      'http://metadata.google.internal/computeMetadata/v1/instance/name', None,
-      {'Metadata-Flavor': 'Google'})
+      'http://metadata.google.internal/computeMetadata/v1/instance/hostname',
+      None, {'Metadata-Flavor': 'Google'})
   try:
     instance = urllib.request.urlopen(req).read().decode('utf-8')
+    #Grab everything excluding the domain part of the hostname
+    instance = instance.split('.')[0]
   except urllib.error.HTTPError as exception:
     raise TurbiniaException(
         'Could not get instance name: {0!s}'.format(exception))
diff --git a/turbinia/state_manager.py b/turbinia/state_manager.py
index b482a1e..9a6fd6c 100644
--- a/turbinia/state_manager.py
+++ b/turbinia/state_manager.py
@@ -237,7 +237,8 @@ class RedisStateManager(BaseStateManager):
   def __init__(self):
     config.LoadConfig()
     self.client = redis.StrictRedis(
-        host=config.REDIS_HOST, port=config.REDIS_PORT, db=config.REDIS_DB)
+        host=config.REDIS_HOST, port=config.REDIS_PORT, db=config.REDIS_DB,
+        socket_timeout=10, socket_keepalive=True, socket_connect_timeout=10)
 
   def _validate_data(self, data):
     return data
diff --git a/turbinia/task_utils.py b/turbinia/task_utils.py
index a361259..47eb08c 100644
--- a/turbinia/task_utils.py
+++ b/turbinia/task_utils.py
@@ -191,28 +191,27 @@ def task_runner(obj, *args, **kwargs):
   Returns:
     Output from TurbiniaTask (should be TurbiniaTaskResult).
   """
-
-  # GKE Specific - do not queue more work if pod places this file
+  obj = task_deserialize(obj)
   if config.TASK_MANAGER.lower() == 'psq':
+    # Late import because this is only needed for PSQ
+    import psq
+    # GKE PSQ Specific - do not queue more work if pod places this file
     if os.path.exists(config.SCALEDOWN_WORKER_FILE):
-      # Late import because this is only needed for PSQ
-      import psq
       raise psq.Retry()
-
-  # Try to acquire lock, timeout and requeue task if the worker
-  # is already processing a task.
-  try:
-    lock = filelock.FileLock(config.LOCK_FILE)
-    with lock.acquire(timeout=0.001):
-      obj = task_deserialize(obj)
-      run = obj.run_wrapper(*args, **kwargs)
-  except filelock.Timeout:
-    if config.TASK_MANAGER.lower() == 'psq':
-      # Late import because this is only needed for PSQ
-      import psq
+    # Try to acquire lock, timeout and requeue task if the worker
+    # is already processing a task.
+    try:
+      lock = filelock.FileLock(config.LOCK_FILE)
+      with lock.acquire(timeout=0.001):
+        run = obj.run_wrapper(*args, **kwargs)
+    except filelock.Timeout:
       raise psq.Retry()
-  # *Always* make sure we release the lock
-  finally:
-    lock.release()
+    # *Always* make sure we release the lock
+    finally:
+      lock.release()
+  # Celery is configured to receive only one Task per worker
+  # so no need to create a FileLock.
+  elif config.TASK_MANAGER.lower() == 'celery':
+    run = obj.run_wrapper(*args, **kwargs)
 
   return run
diff --git a/turbinia/tcelery.py b/turbinia/tcelery.py
index b51bbbf..ccba9fc 100644
--- a/turbinia/tcelery.py
+++ b/turbinia/tcelery.py
@@ -55,12 +55,8 @@ class TurbiniaCelery:
     self.app.conf.update(
         task_default_queue=config.INSTANCE_ID,
         accept_content=['json'],
-        # TODO(ericzinnikas): Without task_acks_late Celery workers will start
-        # on one task and prefetch another (i.e. can result in 1 worker getting
-        # 2 plaso jobs while another worker is free). But enabling this causes
-        # problems with certain Celery brokers (duplicated work).
-        task_acks_late=False,
-        task_track_started=True,
+        task_acks_late=True,
+        worker_cancel_long_running_tasks_on_connection_loss=True,
         worker_concurrency=1,
         worker_prefetch_multiplier=1,
     )
@@ -81,7 +77,11 @@ class TurbiniaKombu(TurbiniaMessageBase):
   def setup(self):
     """Set up Kombu SimpleBuffer"""
     config.LoadConfig()
-    conn = kombu.Connection(config.KOMBU_BROKER)
+    conn = kombu.Connection(
+        config.KOMBU_BROKER, transport_options={
+            'socket_timeout': 10,
+            'socket_keepalive': True
+        })
     if config.KOMBU_DURABLE:
       self.queue = conn.SimpleQueue(name=self.routing_key)
     else:
diff --git a/turbinia/turbiniactl.py b/turbinia/turbiniactl.py
index 1cb3ca2..f0979d2 100644
--- a/turbinia/turbiniactl.py
+++ b/turbinia/turbiniactl.py
@@ -506,14 +506,14 @@ def process_args(args):
   if args.debug_tasks:
     config.DEBUG_TASKS = True
 
-  if config.TASK_MANAGER == 'PSQ':
+  if config.CLOUD_PROVIDER:
     from turbinia.lib import google_cloud
     from libcloudforensics.providers.gcp import forensics as gcp_forensics
 
-  # Enable GCP Stackdriver Logging
-  if config.STACKDRIVER_LOGGING and args.command in ('server', 'psqworker'):
-    google_cloud.setup_stackdriver_handler(
-        config.TURBINIA_PROJECT, args.command)
+    # Enable GCP Stackdriver Logging
+    if config.STACKDRIVER_LOGGING:
+      google_cloud.setup_stackdriver_handler(
+          config.TURBINIA_PROJECT, args.command)
 
   log.info('Turbinia version: {0:s}'.format(__version__))
 
@@ -613,7 +613,7 @@ def process_args(args):
           all_args=all_args)
   elif args.command in ('googleclouddisk', 'googleclouddiskembedded'):
     # Fail if this is a local instance
-    if config.SHARED_FILESYSTEM and not args.force_evidence:
+    if not config.CLOUD_PROVIDER and not args.force_evidence:
       msg = (
           'The evidence type {0:s} is Cloud only, and this instance of '
           'Turbinia is not a cloud instance.'.format(args.command))
diff --git a/turbinia/worker.py b/turbinia/worker.py
index 77832f2..8984944 100644
--- a/turbinia/worker.py
+++ b/turbinia/worker.py
@@ -251,8 +251,13 @@ class TurbiniaCeleryWorker(TurbiniaWorkerBase):
     log.info('Running Turbinia Celery Worker.')
     self._monitoring_setup()
     self._backend_setup()
+    # Disable worker ping checks per amount of signals these generate,
+    # no apparent benefit from having this enabled at the moment.
     self.worker.task(task_utils.task_runner, name='task_runner')
-    argv = ['worker', '--loglevel=info', '--pool=solo']
+    argv = [
+        'worker', '--loglevel=info', '--pool=solo', '--without-gossip',
+        '--without-mingle'
+    ]
     self.worker.start(argv)
 
 


Test patch:
diff --git a/turbinia/e2e/setup-gce.sh b/turbinia/e2e/setup-gce.sh
index 6bb78fb..49f4ffa 100644
--- a/turbinia/e2e/setup-gce.sh
+++ b/turbinia/e2e/setup-gce.sh
@@ -1,6 +1,6 @@
 #!/bin/bash
 # This script will
-# - configure a GCE Ubuntu 18.04 LTS base image for Turbinia e2e testing.
+# - configure a GCE Ubuntu 20.04 LTS base image for Turbinia e2e testing.
 # - execute Terraform to setup the Turbinia environment.
 # Make sure the service account your GCE instance is running under has full API scope
 # access and is project owner for Terraform to function correctly.
diff --git a/turbinia/turbiniactl_test.py b/turbinia/turbiniactl_test.py
index db08103..5d67d46 100644
--- a/turbinia/turbiniactl_test.py
+++ b/turbinia/turbiniactl_test.py
@@ -439,7 +439,7 @@ class TestTurbiniactl(unittest.TestCase):
     self.assertTrue(turbiniactl.process_evidence.called)
 
     # Raise error when running locally
-    config.SHARED_FILESYSTEM = True
+    config.CLOUD_PROVIDER = ''
     with self.assertRaisesRegex(TurbiniaException, 'Cloud only'):
       turbiniactl.process_args([
           'googleclouddiskembedded', '--disk_name', 'disk1,disk2,disk3',