Install Apache Kafka on macOS using Homebrew
Tags: kafka, homebrew, macos
Published: 2023-11-30 12:26:32

---
title: Install Apache Kafka on macOS using Homebrew
published: true
date: 2023-01-01 00:00:00 UTC
tags: kafka,homebrew,macOS,kafka
canonical_url: http://www.javanibble.com/Install-apache-kafka-on-macos-using-homebrew/
---

- [What is Kafka?](#what-is-kafka)
- [Brew Installation Commands](#brew-installation-commands)
- [Directory Structure](#directory-structure)
- [Brew Run Commands](#brew-run-commands)
- [Alternative Commands](#alternative-commands)
- [Testing Installation](#testing-installation)
- [Summary](#summary)

Apache Kafka is a popular open-source stream-processing software platform developed by the Apache Software Foundation. Installing and configuring Kafka can sometimes be daunting, but the process is significantly simplified thanks to Homebrew (or brew), a package manager for macOS.

This blog will guide you through installing Kafka on macOS using Homebrew.

## What is Kafka?

> “Apache Kafka acts as a decoupling layer between source and target systems, allowing source systems to produce data into Kafka, and target systems to consume data from Kafka, making the architecture more scalable and manageable.” [Apache Kafka](https://kafka.apache.org/)

## Brew Installation Commands

Before installing Kafka using Homebrew, you must ensure that Homebrew is installed on your Mac. Once Homebrew is installed, you can use the `brew install` command to install both Kafka and Zookeeper, as Kafka has a dependency on Zookeeper which will be resolved by Homebrew automatically.

_Command: Install Homebrew_

```
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

```

_Command: Install Kafka_

```
$ brew install kafka

```

This command will install both Kafka and Zookeeper, as Zookeeper is a dependency for Kafka.

## Directory Structure

After the installation is complete, the Kafka and Zookeeper binaries, configurations, and logs will be located in different directories based on your Mac’s chip.

**Macs with Intel Processor:**

- Binaries and scripts will be in `/usr/local/bin`
- Kafka configurations will be in `/usr/local/etc/kafka`
- Zookeeper configuration will be in `/usr/local/etc/zookeeper`
- The `log.dirs` config (Kafka data) will be set to `/usr/local/var/lib/kafka-logs`

**Macs with Apple Silicon Processor:**

- Binaries and scripts will be in `opt/homebrew/bin`
- Kafka configurations will be in `opt/homebrew/etc/kafka`
- Zookeeper configuration will be in `opt/homebrew/etc/zookeeper`
- The `log.dirs` config (Kafka data) will be set to `opt/homebrew/var/lib/kafka-logs`

You can modify the Kafka and Zookeeper configuration files to customize your setup.

## Brew Run Commands

Once you have installed Kafka and Zookeeper, you can start them using the brew services command. This command will start Kafka and Zookeeper as background services, so they will continue running even if you close your terminal.

_Command: Start Zookeeper and Kafka_

```
$ brew services start zookeeper
$ brew services start kafka

```

_Command: Stop Zookeeper and Kafka_

```
$ brew services stop kafka
$ brew services stop zookeeper

```

_Command: Restart Zookeeper and Kafka_

```
$ brew services restart kafka
$ brew services restart zookeeper

```

## Alternative Commands

While Homebrew provides a convenient way to manage services, you can also start Zookeeper and Kafka manually using the provided scripts. This involves running the `zookeeper-server-start.sh` and `kafka-server-start.sh` scripts with the appropriate configuration files.

_Command: Start Zookeeper_

```
$ /usr/local/bin/zookeeper-server-start /usr/local/etc/zookeeper/zoo.cfg

```

_Command: Start Kafka_

```
$ /usr/local/bin/kafka-server-start /usr/local/etc/kafka/server.properties

```

Closing the terminal window or using the `Ctrl-C` command will stop the Kafka and Zookeeper services. It is important to stop the services properly to avoid any data corruption or loss.

## Testing Installation

Testing the Kafka installation involves starting the environment, creating a topic, producing events, and consuming events. After successfully performing these tasks, it is important to clean up the environment by deleting the created topic and stopping the Kafka and Zookeeper services to ensure that the system is left in a clean state.

_Command: Start Zookeeper and Kafka_

```
$ brew services start zookeeper
$ brew services start kafka

```

Creating a topic in Kafka can be done using the `kafka-topics -- create` command. After creating the topic, it is important to validate its creation by describing the topic using the command `kafka-topics --describe`, which will display the details of the created topic.

_Command: Create a Topic_

```
$ kafka-topics --create --topic quickstart-events --bootstrap-server localhost:9092
$ kafka-topics --describe --topic quickstart-events --bootstrap-server localhost:9092

```

Producing new events in Kafka can be done using the `kafka-console-producer` command. This will send a `Hello World` event to the Kafka topic you created.

_Command: Produce Events_

```
$ kafka-console-producer.sh --topic quickstart-events --bootstrap-server localhost:9092
> Hello World 1.
> Hello World 2.

```

You can stop the producer client with `Ctrl-C` at any time.

Consuming the events in Kafka can be done using the `kafka-console-consumer` command. This will retrieve the `Hello World` event from the Kafka topic you created.

_Command: Consume Events_

```
$ kafka-console-consumer --topic quickstart-events --from-beginning --bootstrap-server localhost:9092
> Hello World 1.
> Hello World 2.

```

You can stop the consumer client with `Ctrl-C` at any time.

Cleaning up the environment after testing the Kafka installation involves several steps. First, you need to delete the topic you created using the `kafka-topics` command, then you need to shut down Kafka and Zookeeper using the `brew services stop` commands, and finally, you need to delete any data of your local Kafka environment, including any events you have created along the way, by removing the Kafka logs and Zookeeper data directories that were specified in the configuration files.

_Command: Environment Cleanup_

```
$ kafka-topics --delete --topic quickstart-events --bootstrap-server localhost:9092
$ brew services stop kafka
$ brew services stop zookeeper
$ rm -rf /tmp/kafka-logs /tmp/zookeeper /tmp/kraft-combined-logs

```

## Summary

Installing Kafka on macOS is straightforward using Homebrew. With just a few commands, you can have Kafka and Zookeeper up and running on your machine.

Happy streaming!