Integrating ClickHouse with Databricks

### Secret management[​](#secret-management "Direct link to Secret management")

Use Databricks secret scopes to securely store ClickHouse credentials:

```
# Access secrets
password = dbutils.secrets.get(scope="clickhouse", key="password")
```

For setup instructions, see the Databricks [Secret management documentation](https://docs.databricks.com/aws/en/security/secrets/).

### ClickHouse Cloud connection[​](#clickhouse-cloud "Direct link to ClickHouse Cloud connection")

When connecting to ClickHouse Cloud from Databricks:

1. Use **HTTPS protocol** (`protocol: https`, `http_port: 8443`)
2. Enable **SSL** (`ssl: true`)

## Examples[​](#examples "Direct link to Examples")

### Complete workflow example[​](#workflow-example "Direct link to Complete workflow example")

- Python- Scala

```
from pyspark.sql import SparkSession
from pyspark.sql.functions import col

# Initialize Spark with ClickHouse connector
spark = SparkSession.builder \
    .config("spark.jars.packages", "com.clickhouse.spark:clickhouse-spark-runtime-3.4_2.12:0.9.0") \
    .getOrCreate()

# Read from ClickHouse
df = spark.read \
    .format("clickhouse") \
    .option("host", "your-host.clickhouse.cloud") \
    .option("protocol", "https") \
    .option("http_port", "8443") \
    .option("database", "default") \
    .option("table", "source_table") \
    .option("user", "default") \
    .option("password", dbutils.secrets.get(scope="clickhouse", key="password")) \
    .option("ssl", "true") \
    .load()

# Transform data
transformed_df = df.filter(col("status") == "active")

# Write to ClickHouse
transformed_df.write \
    .format("clickhouse") \
    .option("host", "your-host.clickhouse.cloud") \
    .option("protocol", "https") \
    .option("http_port", "8443") \
    .option("database", "default") \
    .option("table", "target_table") \
    .option("user", "default") \
    .option("password", dbutils.secrets.get(scope="clickhouse", key="password")) \
    .option("ssl", "true") \
    .option("order_by", "id") \
    .mode("append") \
    .save()
```

```
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.col

// Initialize Spark with ClickHouse connector
val spark = SparkSession.builder
  .config("spark.jars.packages", "com.clickhouse.spark:clickhouse-spark-runtime-3.4_2.12:0.9.0")
  .getOrCreate()

// Read from ClickHouse
val df = spark.read
  .format("clickhouse")
  .option("host", "your-host.clickhouse.cloud")
  .option("protocol", "https")
  .option("http_port", "8443")
  .option("database", "default")
  .option("table", "source_table")
  .option("user", "default")
  .option("password", dbutils.secrets.get(scope="clickhouse", key="password"))
  .option("ssl", "true")
  .load()

// Transform data
val transformedDF = df.filter(col("status") === "active")

// Write to ClickHouse
transformedDF.write
  .format("clickhouse")
  .option("host", "your-host.clickhouse.cloud")
  .option("protocol", "https")
  .option("http_port", "8443")
  .option("database", "default")
  .option("table", "target_table")
  .option("user", "default")
  .option("password", dbutils.secrets.get(scope="clickhouse", key="password"))
  .option("ssl", "true")
  .option("order_by", "id")
  .mode("append")
  .save()
```

## Related documentation[​](#related "Direct link to Related documentation")

- [Spark Native Connector Guide](/docs/integrations/apache-spark/spark-native-connector) - Complete connector documentation
- [TableProvider API Documentation](/docs/integrations/apache-spark/spark-native-connector#using-the-tableprovider-api) - Format-based access details
- [Catalog API Documentation](/docs/integrations/apache-spark/spark-native-connector#register-the-catalog-required) - Catalog-based access details

- [API Selection for Databricks](#api-selection)- [Installation on Databricks](#installation)
    - [Option 1: Upload JAR via Databricks UI](#installation-ui)- [Option 2: Install via Databricks CLI](#installation-cli)- [Option 3: Maven Coordinates (Recommended)](#installation-maven)- [Using TableProvider API](#tableprovider-api)
      - [Reading data](#reading-data-table-provider)- [Writing data](#writing-data-unity)- [Databricks-specific considerations](#considerations)
        - [Access mode requirements](#access-mode)- [Secret management](#secret-management)- [ClickHouse Cloud connection](#clickhouse-cloud)- [Examples](#examples)
          - [Complete workflow example](#workflow-example)- [Related documentation](#related)

Was this page helpful?