Project: Apache Heron (App Definition and Development / Streaming & Messaging)
File: schedulers-k8s-with-helm.md

id: version-0.20.2-incubating-schedulers-k8s-with-helm title: Kubernetes with Helm sidebar_label: Kubernetes with Helm original_id: schedulers-k8s-with-helm <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> If you'd prefer to install Heron on Kubernetes without using the package manager, see the document. is an open source package manager for that enables you to quickly and easily install even the most complex software systems on Kubernetes. Heron has a Helm that you can use to install Heron on Kubernetes using just a few commands. The chart can be used to install Heron on the following platforms: (the default) In order to install Heron on Kubernetes using Helm, you'll need to have an existing Kubernetes cluster on one of the supported (which includes installations). In order to get started, you need to install Helm on your machine. Installation instructions for and are below. You can install Helm on macOS using : ```bash $ brew install kubernetes-helm ``` You can install Helm on Linux using a simple installation script: ```bash $ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 $ chmod 700 get_helm.sh $ ./get_helm.sh ``` To use Helm with Kubernetes, you need to first make sure that is using the right configuration context for your cluster. To check which context is being used: ```bash $ kubectl config current-context ``` Once you've installed the Helm client on your machine and gotten Helm pointing to your Kubernetes cluster, you need to make your client aware of the `heron-charts` Helm repository, which houses the chart for Heron: ```bash $ helm repo add heron-charts https://storage.googleapis.com/heron-charts "heron-charts" has been added to your repositories ``` Create a namespace to install into: ```bash $ kubectl create namespace heron ``` Now you can install the Heron package: ```bash $ helm install heron-charts/heron -g -n heron ``` This will install Heron and provide the installation in the `heron` namespace (`-n`) with a random name (`-g`) like `jazzy-anaconda`. To provide the installation with a name, such as `heron-kube`: ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron ``` The default platform for running Heron on Kubernetes is . To specify a different platform, you can use the `--set platform=PLATFORM` flag. Here's an example: ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron \ --set platform=gke ``` The available platforms are: Platform | Tag :--|: | `minikube` | `gke` | `aws` | `baremetal` To run Heron on Minikube, you need to first . Once Minikube is installed, you can start it by running `minikube

Please note, however, that Heron currently requires the following resources: 7 GB of memory 5 CPUs 20 GB of disk space To start up Minikube with the minimum necessary resources: ```bash $ minikube start \ --memory=7168 \ --cpus=5 \ --disk-size=20g ``` Once Minikube is running, you can then install Heron in one of two ways: Create a namespace to install into: ```bash $ kubectl create namespace heron ``` ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron $ helm install heron-kube heron-charts/heron \ --namespace heron \ --set platform=minikube ``` The resources required to run Heron on vary based on your use case. To run a basic Heron cluster intended for development and experimentation, you'll need at least: 3 nodes machines To create a cluster with those resources using the tool: ```bash $ gcloud container clusters create heron-gke-dev-cluster \ --num-nodes=3 \ --machine-type=n1-standard-2 ``` For a production-ready cluster you'll want a larger cluster with: at least* 8 nodes machines (preferably the latter) To create such a cluster: ```bash $ gcloud container clusters create heron-gke-prod-cluster \ --num-nodes=8 \ --machine-type=n1-standard-8 ``` Once the cluster has been successfully created, you'll need to install that cluster's credentials locally so that they can be used by . You can do this in just one command: ```bash $ gcloud container clusters get-credentials heron-gke-dev-cluster # or heron-gke-prod-cluster ``` Create a namespace to install into: ```bash $ kubectl create namespace heron ``` Now you can install Heron: ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron \ --set platform=gke ``` Helm enables you to supply sets of variables via YAML files. There are currently a handful of different resource configurations that can be applied to your Heron on GKE cluster upon installation: Configuration | Description :-|:-- | Smaller Heron cluster intended for basic testing, development, and experimentation | Closer geared for production usage To apply the `small` configuration, for example: ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron \ --set platform=gke \ --values https://raw.githubusercontent.com/apache/incubator-heron/master/deploy/kubernetes/gcp/small.yaml ``` To run Heron on Kubernetes on Amazon Web Services (AWS), you'll need to ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron \ --set platform=aws ``` You can make Heron to use S3 to distribute the user topologies. First you need to set up a S3 bucket and configure an IAM user with enough permissions over it. Get access keys for the user. Then you can deploy Heron like this: ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron \ --set platform=aws \ --set uploader.class=s3 \ --set uploader.s3Bucket=heron \ --set uploader.s3PathPrefix=topologies \ --set uploader.s3AccessKey=XXXXXXXXXXXXXXXXXXXX \ --set uploader.s3SecretKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX \ --set uploader.s3Region=us-west-1 ``` To run Heron on a bare metal Kubernetes cluster: ```bash $ helm install heron-kube heron-charts/heron \ --namespace heron \ --set platform=baremetal ``` When setting the `heron` CLI configuration, make sure that the cluster name matches the name of the Helm installation. This can be either the name auto-generated by Helm or the name you supplied via the `--name` flag upon installation (in some of the examples above, the `heron-kubernetes` name was used). Make sure to adjust the name accordingly if necessary. Once all of the components have been successfully started up, you need to open up a proxy port to your Kubernetes cluster using the command: ```bash $ kubectl proxy -p 8001 ``` Note: All of the following Kubernetes specific urls are valid with the Kubernetes 1.10.0 release. Now, verify that the Heron API server running on Minikube is available using curl: ```bash $ curl http://localhost:8001/api/v1/namespaces/default/services/heron-kubernetes-apiserver:9000/proxy/api/v1/version ``` You should get a JSON response like this: ```json { "heron.build.git.revision" : "ddbb98bbf173fb082c6fd575caaa35205abe34df", "heron.build.git.status" : "Clean", "heron.build.host" : "ci-server-01", "heron.build.time" : "Sat Mar 31 09:27:19 UTC