Step-by-Step: Deploy the most advanced Kafka Console on AWS in minutes
Tags: kafka, tutorial, devops, aws
Published: 2024-02-23 12:45:44

_Launching Conduktor in minutes using CloudFormation and ECS Fargate._

## Overview
Utilizing a console for managing your Kafka clusters, checking topic configs, monitoring lag, and analyzing your Kafka data can save you a lot of time. You get a complete picture through one pane of glass, as opposed to wrangling with a multitude of CLIs, APIs and devtools.

[Conduktor](https://www.conduktor.io/) is the most advanced console for getting insight into your Kafka ecosystem. It’s a vendor agnostic solution, meaning it works with Confluent, AWS MSK, Redpanda and all other Kafka protocol compliant providers. It also supports Confluent Schema Registry and AWS Glue, as well as Kafka Connect and ksqlDB.

This guide will show you how to get a free version of Conduktor up and running alongside all dependencies on ECS in minutes. Using CloudFormation, the process is dead simple and requires only a few clicks. If you prefer Kubernetes, try our [Helm chart](https://docs.conduktor.io/platform/get-started/installation/get-started/kubernetes/).

![Conduktor Console Kafka UI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mejg025flfco0vqjlks9.png)

***

## Networking & Architecture

The CloudFormation script will deploy an RDS instance for storing metadata such as user preferences, resource tags, users, groups and permissions. 

The Conduktor service will be exposed on a public IP address by default, though you can edit this if it's not suitable for your need.

![Networking & Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ue2022jhfsmfe8qn6sd7.png)

***

## Deployment Steps

_While this guide will help you get started, you may need to make additional configurations to ensure your deployment is [production-ready](https://docs.conduktor.io/platform/get-started/installation/hardware/#production-requirements)._

- To start the deployment, please go to https://aws.amazon.com/cloudformation/getting-started/

- Copy/sync from github using one of the following CloudFormation templates:
    - [Full](https://github.com/conduktor/quickstart-conduktor-cloudformation/blob/main/templates/CDK-full-template.yaml) - use this to deploy Conduktor and to spin up the infrastructure needed to run it.
    - [Lite](https://github.com/conduktor/quickstart-conduktor-cloudformation/blob/main/templates/CDK-lite-template.yaml) - use this to deploy Conduktor on existing infrastructure (note you will need an ECS Cluster and an RDS database).

![CloudFormation Template](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7oiijy2n0t2ej5a4cstv.png)

- Wait for the stack to be created. This should take 5-10 minutes.

- Go to Cluster → Service → Task → Conduktor Console → Network Bindings and click on the address.

![Network Binding](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fngfvp3neefln5lg7gxi.png)

- You can now onboard onto Console. The first step is to add your Kafka cluster configuration.

![Kafka UI](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/te0nrmeouzbz94wh8f2h.png)

- In the cluster configuration screen, add your Kafka clusters bootstrap server and authentication details.
 - If you are using Schema registry, Kafka Connect or ksqlDB, you should also add these details via the tabs highlighted below.

![Bootstrap Server](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rs7dk1xw4frctre0ii0q.png)

- Final step! Invite your team mates to collaborate with you from the Users & Groups screen.
    - Note you can configure SSO (LDAP, OIDC) or add users via basic authentication.

![SSO IDP](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r17fl4zpsil8ne3pjmb2.png)

***

### That's a wrap!

You're all ready to go with the most powerful Kafka Console on the market. Start drilling down into your Kafka data, managing your subjects, connectors, consumer groups, service accounts and more!

![Topic Message Filter](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nnr595wn639az9l0mxdw.png)

Refer to the [documentation](https://docs.conduktor.io/platform/get-started/installation/get-started/CloudFormation/) for more detailed steps and other deployment options such as [Docker](https://docs.conduktor.io/platform/get-started/installation/get-started/docker/) and [Helm](https://docs.conduktor.io/platform/get-started/installation/get-started/kubernetes/).