How to quickly test connectivity to your Azure Event Hubs for Kafka cluster, without writing any code
Tags: azure, productivity, showdev
Published: 2019-09-13 12:21:01

You have a shiny new [Kafka enabled broker on Azure Event Hubs](https://docs.microsoft.com/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview?WT.mc_id=devto-blog-abhishgu) and want to quickly test it out without writing cumbersome client (producer and consumer) code. Try out the instructions in this post and you should (hopefully) have everything setup and sanity tested in ~ 10 minutes.

![](https://thepracticaldev.s3.amazonaws.com/i/rilzkcc6a30c1l4us45b.gif)

Clone [this GitHub repo](https://github.com/abhirockzz/azure-eventhubs-kafka-cli-quickstart) and change to the correct directory

    git clone https://github.com/abhirockzz/azure-eventhubs-kafka-cli-quickstart
    cd azure-eventhubs-kafka-cli-quickstart

> You can either choose to install the [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli?view=azure-cli-latest&WT.mc_id=devto-blog-abhishgu) if you don't have it already (should be quick!) or just use the [Azure Cloud Shell](https://azure.microsoft.com/features/cloud-shell/?WT.mc_id=devto-blog-abhishgu) from your browser.

### Create your Kafka enabled Event Hubs cluster

If you have a cluster already, skip this and go to the *"Get what you need to connect to the cluster"* section

Set some variables to avoid repetition

    AZURE_SUBSCRIPTION=[to be filled]
    AZURE_RESOURCE_GROUP=[to be filled]
    AZURE_LOCATION=[to be filled]
    EVENT_HUBS_NAMESPACE=[to be filled]
    EVENT_HUB_NAME=[to be filled]

Create the resource group if you don't have one already

    az account set --subscription $AZURE_SUBSCRIPTION
    az group create --name $AZURE_RESOURCE_GROUP --location $AZURE_LOCATION

Create an [Event Hubs namespace](https://docs.microsoft.com/azure/event-hubs/event-hubs-features?WT.mc_id=devto-blog-abhishgu#namespace) (similar to a Kafka Cluster)

    az eventhubs namespace create --name $EVENT_HUBS_NAMESPACE --resource-group $AZURE_RESOURCE_GROUP --location $AZURE_LOCATION --enable-kafka true --enable-auto-inflate false

And then create an Event Hub (same as a Kafka topic)

    az eventhubs eventhub create --name $EVENT_HUB_NAME --resource-group $AZURE_RESOURCE_GROUP --namespace-name $EVENT_HUBS_NAMESPACE --partition-count 3

### Get what you need to connect to the cluster

Get the connection string and credentials for your cluster

> For details, read [how Event Hubs uses Shared Access Signatures for authorization](https://docs.microsoft.com/azure/event-hubs/authorize-access-shared-access-signature?WT.mc_id=devto-blog-abhishgu)

Start by getting the Event Hub rule/policy name

    EVENT_HUB_AUTH_RULE_NAME=$(az eventhubs namespace authorization-rule list --resource-group $AZURE_RESOURCE_GROUP --namespace-name $EVENT_HUBS_NAMESPACE | jq '.[0].name' | sed "s/\"//g")

And, then make use of the rule name to extract the connection string

    az eventhubs namespace authorization-rule keys list --resource-group $AZURE_RESOURCE_GROUP --namespace-name $EVENT_HUBS_NAMESPACE --name $EVENT_HUB_AUTH_RULE_NAME | jq '.primaryConnectionString'

> [This information is sensitive - please excercise caution](https://docs.microsoft.com/azure/event-hubs/authorize-access-shared-access-signature?WT.mc_id=devto-blog-abhishgu#best-practices-when-using-sas)

Copy the result of above command in the `password` field within `jaas.conf` file.


    KafkaClient {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="$ConnectionString"
        password=enter-connection-string-here;
    };

> DO NOT remove the trailing `;` in `jaas.conf`


### Connect to the cluster using Kafka CLI

> I am assuming that you already have a Kafka setup (local or elsewhere) - the Kafka CLI is bundled along with it. If not, it shouldn't take too long to set it up - just [download, unzip](https://kafka.apache.org/downloads) and you're ready to go!

On your local machine, use a new terminal to start a **Kafka Consumer** - set the required variables first

    EVENT_HUBS_NAMESPACE=[to be filled]
    EVENT_HUB_NAME=[to be filled]
    export KAFKA_OPTS="-Djava.security.auth.login.config=jaas.conf"
    KAFKA_INSTALL_HOME=[to be filled] e.g. /Users/jdoe/kafka_2.12-2.3.0/

Start consuming

    $KAFKA_INSTALL_HOME/bin/kafka-console-consumer.sh --topic $EVENT_HUB_NAME --bootstrap-server $EVENT_HUBS_NAMESPACE.servicebus.windows.net:9093 --consumer.config client_common.properties


Use another terminal to start a **Kafka Producer** - set the required variables first

    EVENT_HUBS_NAMESPACE=[to be filled]
    EVENT_HUB_NAME=[to be filled]
    export KAFKA_OPTS="-Djava.security.auth.login.config=jaas.conf"
    KAFKA_INSTALL_HOME=[to be filled] e.g. /Users/jdoe/kafka_2.12-2.3.0/

Start the producer


    $KAFKA_INSTALL_HOME/bin/kafka-console-producer.sh --topic $EVENT_HUB_NAME --broker-list $EVENT_HUBS_NAMESPACE.servicebus.windows.net:9093 --producer.config client_common.properties

You will get a prompt and you can start entering values e.g.

    > foo
    > bar
    > baz
    > john
    > doe

Switch over to the consumer terminal to confirm you have got the messages! 

![](https://media.giphy.com/media/QN6NnhbgfOpoI/giphy.gif)


### For your reference...

Here is a handy list of all the Event Hubs related CLI commands which were used

- [az eventhubs namespace create](https://docs.microsoft.com/cli/azure/eventhubs/namespace?view=azure-cli-latest&WT.mc_id=devto-blog-abhishgu#az-eventhubs-namespace-create) - Creates the EventHubs Namespace.
- [az eventhubs eventhub create](https://docs.microsoft.com/cli/azure/eventhubs/eventhub?view=azure-cli-latest&WT.mc_id=devto-blog-abhishgu#az-eventhubs-eventhub-create) - Creates the EventHubs Eventhub.
- [az eventhubs namespace authorization-rule list](https://docs.microsoft.com/cli/azure/eventhubs/namespace/authorization-rule?view=azure-cli-latest&WT.mc_id=devto-blog-abhishgu#az-eventhubs-namespace-authorization-rule-list) - Shows the list of Authorizationrule by Namespace.
- [az eventhubs namespace authorization-rule keys list](https://docs.microsoft.com/cli/azure/eventhubs/namespace/authorization-rule/keys?view=azure-cli-latest&WT.mc_id=devto-blog-abhishgu#az-eventhubs-namespace-authorization-rule-keys-list) - Shows the connection strings for namespace.

I really hope you enjoyed and learned something from this article! Please like and follow if you did. Happy to get feedback via [@abhi_tweeter](https://twitter.com/abhi_tweeter) or just drop a comment.