Redpanda: Quickstart to selfhost Redpanda
Tags: kafka, kubernetes, redpanda, data
Published: 2023-03-28 11:00:40

⏰ _Estimated Time: 4 minutes_

This guide covers the steps to self host Redpanda on your Kubernetes cluster using Argonaut.

[Redpanda](https://redpanda.com/) is a distributed event streaming platform providing infrastructure for real-time data. It is designed to be fast, scalable, and easy to use. It is built on top of Apache Kafka and is fully compatible with the Kafka ecosystem, while being packaged as a single binary without the need for ZooKeeper or a separate control plane.

This makes for a lightweight and easily maintainable deployment. It provides a number of features that make it well-suited for modern streaming applications, including support for transactions, and a high-performance storage engine.

We will deploy both Redpanda and the Redpanda console.

## Prerequisites for installation

- [Argonaut](https://ship.argonaut.dev/) account with an environment created
- AWS or GCP account connected to Argonaut
- EKS or GKE cluster provisioned by Argonaut or any other imported Kubernetes cluster

_Note:_ This installation requires Kubernetes 1.21+. Argonaut automatically maintains Kubernetes versions.

## Setup Library App in Argonaut

You can add Redpanda as a custom app from Library to Argonaut. This will then be available in your cluster under **Add-Ons**.

> 💡 Each Redpanda broker runs on its own worker node and requires a minimum of 3 nodes. This can be changed in Helm chart configuration's `podAntiAffinity` rules by setting `podAntiAffinity.type: soft`.

1. Click on the `Application +` and the `From Library` button.  
  ![Select Library Application](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ju8txskd1xqka0tfcr44.png)
1. Choose `Custom-Apps` under configuration.
1. Ensure the selected environment is correct, then select the cluster you want to deploy the agent to. Important to ensure that the environment and region of this node is the same as all other nodes.
1. Set the configuration to Custom Apps (Helm)
   1. Set namespace as the same name as the cluster (that's where your apps are deployed by default)
   1. Release Name as `redpanda`
   1. Chart Name as `redpanda`
   1. Repo Url as `https://charts.redpanda.com`
1. The Helm version will be updated automatically.
   ![Deployment information](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/beu9d3dx9ie0n7xq66r5.png)

1. Make the following changes to `values.yaml` file.
      ```yaml
      # Logging
      logging:
        # Log level
        # Valid values (from least to most logging) are warn, info, debug, trace
        logLevel: warn
      ```
1. Notes:
   1. Multiple `redpanda` instances running in the same cluster will need manual port tweaks to avoid conflicts. This is supported by the chart.
   1. Secrets must be used for passwords in production environments.
   1. Optionally, enable [SASL authentication](https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer) by editing this in the `values.yaml` file.

      ```yaml
      # Authentication
      auth:
        sasl:
          enabled: true
          users:
            - name: superuser
              password: <secretpassword>
      ```

1. Click on `Install` to deploy the app.

## Accessing Redpanda

The Kafka interface is exposed on port 9094 by default (`listeners.kafka.external.default.port`). The Redpanda console is exposed on port 9644 within the cluster (`listeners.admin.port`).

### Components installed

The Redpanda Helm chart will install the following components into your Kubernetes cluster:

- [A StatefulSet](https://docs.redpanda.com/docs/deploy/deployment-option/self-hosted/kubernetes/eks-guide/#statefulset) with three Pods.
- [One PersistentVolumeClaim](https://docs.redpanda.com/docs/deploy/deployment-option/self-hosted/kubernetes/eks-guide/#persistentvolumeclaims) for each Pod, each with a capacity of 20Gi.
- [A headless ClusterIP Service and a NodePort Service](https://docs.redpanda.com/docs/deploy/deployment-option/self-hosted/kubernetes/eks-guide/#service) for each Kubernetes node that runs a Redpanda broker.