Creating your first Kafka ClickPipe

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/integrations/data-ingestion/clickpipes/kafka/01_create-kafka-clickpipe.md)

> In this guide, we will walk you through the process of creating your first Kafka ClickPipe.

Navigate to data sources

## Navigate to data sources[​](#1-load-sql-console "Direct link to Navigate to data sources")

Select the `Data Sources` button on the left-side menu and click on "Set up a ClickPipe".

Select a data source

## Select a data source[​](#2-select-data-source "Direct link to Select a data source")

Select your Kafka data source from the list.

Configure the data source

## Configure the data source[​](#3-configure-data-source "Direct link to Configure the data source")

Fill out the form by providing your ClickPipe with a name, a description (optional), your credentials, and other connection details.

Configure a schema registry (optional)

## Configure a schema registry (optional)[​](#4-configure-your-schema-registry "Direct link to Configure a schema registry (optional)")

A valid schema is required for Avro streams. See [Schema registries](/docs/integrations/clickpipes/kafka/schema-registries) for more details on how to configure a schema registry.

Configure a reverse private endpoint (optional)

## Configure a reverse private endpoint (optional)[​](#5-configure-reverse-private-endpoint "Direct link to Configure a reverse private endpoint (optional)")

Configure a Reverse Private Endpoint to allow ClickPipes to connect to your Kafka cluster using AWS PrivateLink.
See our [AWS PrivateLink documentation](/docs/integrations/clickpipes/aws-privatelink) for more information.

Configure SSH tunneling (optional)

## Configure SSH tunneling (optional)[​](#6-configure-ssh-tunneling "Direct link to Configure SSH tunneling (optional)")

You can use SSH tunneling if your Kafka broker isn't publicly accessible. Instead of connecting directly, ClickPipes establishes an SSH connection to a bastion host (a server in your network that is publicly accessible) and then forwards traffic through it to your Kafka broker on your private network.

1. Enable the "SSH Tunnel" toggle.
2. Fill in the SSH connection details:
   - **SSH Host**: The hostname or IP address of your bastion host — this is the publicly accessible server that acts as a gateway into your private network.
   - **SSH Port**: The port for SSH on the bastion host (default `22`).
   - **SSH User**: The username to authenticate with on the bastion host.

3. To use Key-based authentication, click on "Revoke and regenerate key pair" to generate a new key pair and copy the generated public key to your SSH server under `~/.ssh/authorized_keys`.
4. Click on "Verify Connection" to verify the connection.

Note

Make sure to whitelist [ClickPipes IP addresses](/docs/integrations/clickpipes#list-of-static-ips) in your firewall rules for the SSH bastion host so that ClickPipes can establish the SSH tunnel.

Select your topic

## Select your topic[​](#7-select-your-topic "Direct link to Select your topic")

Select your topic and the UI will display a sample document from the topic.

Configure your destination table

## Configure your destination table[​](#8-configure-your-destination-table "Direct link to Configure your destination table")

In the next step, you can select whether you want to ingest data into a new ClickHouse table or reuse an existing one. Follow the instructions in the screen to modify your table name, schema, and settings. You can see a real-time preview of your changes in the sample table at the top.

You can also customize the advanced settings using the controls provided

Configure permissions

## Configure permissions[​](#9-configure-permissions "Direct link to Configure permissions")

ClickPipes will create a dedicated user for writing data into a destination table. You can select a role for this internal user using a custom role or one of the predefined role:

- `Full access`: with the full access to the cluster. It might be useful if you use Materialized View or Dictionary with the destination table.
- `Only destination table`: with the `INSERT` permissions to the destination table only.

Complete setup

## Complete setup[​](#10-complete-setup "Direct link to Complete setup")

Clicking on "Create ClickPipe" will create and run your ClickPipe. It will now be listed in the Data Sources section.

- [Navigate to data sources](#1-load-sql-console)- [Select a data source](#2-select-data-source)- [Configure the data source](#3-configure-data-source)- [Configure a schema registry (optional)](#4-configure-your-schema-registry)- [Configure a reverse private endpoint (optional)](#5-configure-reverse-private-endpoint)- [Configure SSH tunneling (optional)](#6-configure-ssh-tunneling)- [Select your topic](#7-select-your-topic)- [Configure your destination table](#8-configure-your-destination-table)- [Configure permissions](#9-configure-permissions)- [Complete setup](#10-complete-setup)

Was this page helpful?