ClickHouse/clickhouse-docs-embeddings huggingface dataset:
Knowledge Base

This guide shows you how to collect and draw query traces with self-managed ClickHouse using either built-in methods or using Grafana. This is particularly useful when you're working with complex queries and need to understand the internal execution mechanics beyond what EXPLAIN provides.

Dec 30, 2025 · 4 min read

A HAR (HTTP Archive) file captures the network activity in your browser. It can help our support team diagnose slow page loads, failed requests, or other network issues.

Nov 11, 2025 · 2 min read

Learn how to profile ClickHouse using LLVM's XRay instrumentation profiler, visualize traces, and analyze performance.

Nov 13, 2024 · 4 min read

Example of mapping Windows Active Directory security groups to ClickHouse roles

May 20, 2024 · 6 min read

Commands to build LLVM and clang on Linux.

Apr 22, 2024 · One min read

Learn how to start, stop, and resume a ClickHouse Cloud service using API endpoints and cURL commands.

Jun 7, 2023 · 2 min read

Learn about schema migration tools for ClickHouse and how to manage changing database schemas over time.

Jun 7, 2023 · 7 min read

Learn how to install and test ClickHouse on Windows 10 using WSL 2. Includes setup, troubleshooting, and running a test environment.

Apr 24, 2023 · 3 min read

Learn how to quickly recreate a small table and its data across different terminals using copy/paste for development environments.

Mar 24, 2023 · 2 min read
title_wip

Learn how to use ClickHouse for vector search, including storing embeddings and searching with distance functions like cosine similarity.

Oct 26, 2023 · 2 min read

Learn how to create and query parameterized views in ClickHouse for dynamic data slicing based on query-time parameters.

Aug 1, 2023 · 2 min read

ClickHouse is popular for logs and metrics analysis because of the real-time analytics capabilities provided. Ready to find out more?

May 10, 2023 · One min read

It's Pi Day! Let's calculate pi using ClickHouse SQL

Mar 14, 2023 · 3 min read

The short answer is \*\*"no"\*\*. The key-value workload is among top positions in the list of cases when \*\*NOT\*\* to use ClickHouse.

Sep 1, 2021 · 2 min read

Being an open-source product makes this question not so straightforward to answer. You do not have to tell anyone if you want to start using ClickHouse, you just go grab source code or pre-compiled packages.

Sep 1, 2021 · 2 min read
title_wip

ClickHouse is a generic data storage solution for OLAP workloads, while there are many specialized time-series database management systems.


## Answer[​](#answer "Direct link to Answer")

Note

Please see the blog [Working with Time series data in ClickHouse](https://clickhouse.com/blog/working-with-time-series-data-and-functions-ClickHouse) for additional examples of using ClickHouse for time series analysis.\_

ClickHouse is a generic data storage solution for [OLAP](https://clickhouse.com/docs/faq/general/olap) workloads, while there are many specialized time-series database management systems. Nevertheless, ClickHouse’s [focus on query execution speed](https://clickhouse.com/docs/faq/general/why-clickhouse-is-so-fast) allows it to outperform specialized systems in many cases. There are many independent benchmarks on this topic out there, so we’re not going to conduct one here. Instead, let’s focus on ClickHouse features that are important to use if that’s your use case.

First of all, there are **[specialized codecs](https://clickhouse.com/docs/sql-reference/statements/create/table#specialized-codecs)** which make typical time-series. Either common algorithms like `DoubleDelta` and `Gorilla` or specific to ClickHouse like `T64`.

Second, time-series queries often hit only recent data, like one day or one week old. It makes sense to use servers that have both fast NVMe/SSD drives and high-capacity HDD drives. ClickHouse [TTL](https://clickhouse.com/docs/engines/table-engines/mergetree-family/mergetree/##table_engine-mergetree-multiple-volumes) feature allows to configure keeping fresh hot data on fast drives and gradually move it to slower drives as it ages. Rollup or removal of even older data is also possible if your requirements demand it.

Even though it’s against ClickHouse philosophy of storing and processing raw data, you can use [materialized views](https://clickhouse.com/docs/sql-reference/statements/create/view) to fit into even tighter latency or costs requirements.

## Related Content[​](#related-content "Direct link to Related Content")

- Blog: [Working with time series data in ClickHouse](https://clickhouse.com/blog/working-with-time-series-data-and-functions-ClickHouse)

Sep 1, 2021 · 2 min read
title_wip

Troubleshooting access issues with ClickHouse Cloud services, including IP Access List configuration


## I am unable to access a ClickHouse Cloud service[​](#i-am-unable-to-access-a-clickhouse-cloud-service "Direct link to I am unable to access a ClickHouse Cloud service")

If you are seeing an error message like one of these, your IP Access List may be denying access:

```
curl: (35) error:02FFF036:system library:func(4095):Connection reset by peer
```

or

```
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to HOSTNAME.clickhouse.cloud:8443
```

or

```
Code: 210. DB::NetException: SSL connection unexpectedly closed (e46453teek.us-east-2.aws.clickhouse-staging.com:9440). (NETWORK_ERROR)
```

Check the [IP Access List](/docs/cloud/security/setting-ip-filters), if you are attempting to connect from outside the allowed list then your connection will fail.

Jul 2, 2024 · One min read
title_wip

Recommended configuration:

Set `min_level_for_full_part_storage = 2` for cloud deployments.
This ensures that:

- Level 0 parts (initial inserts) use packed storage
- Level 1 parts continue using packed storage
- Only parts at level 2 and above use full storage format

Tip

Avoid this setting for tables that receive very large but infrequent writes where insufficient merges occur, as large initial writes may benefit from full part storage immediately.

## Combining part types and storage formats[​](#combining-part-types-and-storage-formats "Direct link to Combining part types and storage formats")

These two concepts are orthogonal and you can have any combination, depending on if you are using ClickHouse Cloud or Open Source ClickHouse:

|  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Combination Use case|  |  |  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | --- | --- | | Wide + Full Large parts on local storage (default in open source)|  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | | Wide + Packed Large parts on cloud storage|  |  |  |  | | --- | --- | --- | --- | | Compact + Full Small parts on local storage|  |  | | --- | --- | | Compact + Packed Small parts on cloud storage | | | | | | | | | |

## Querying part information[​](#querying-part-information "Direct link to Querying part information")

You can inspect the part type and part storage type of existing parts using the [`system.parts`](/docs/operations/system-tables/parts) table:

```
SELECT
    part_type,
    part_storage_type,
    max(level),
    count(),
    formatReadableSize(max(data_uncompressed_bytes)),
    formatReadableSize(min(data_uncompressed_bytes))
FROM system.parts
WHERE (database != 'system') AND active
GROUP BY
    1,
    2
ORDER BY
    1 ASC,
    2 ASC
```

You will see something like this:

```
   ┌─part_type─┬─part_storage_type─┬─max(level)─┬─count()─┬─formatReadab⋯sed_bytes))─┬─formatReadab⋯sed_bytes))─┐
1. │ Compact   │ Full              │      12688 │    2456 │ 1023.87 MiB              │ 23.78 MiB                │
2. │ Compact   │ Packed            │      97383 │   13748 │ 127.77 MiB               │ 1.00 B                   │
3. │ Wide      │ Full              │       7642 │    2000 │ 1.38 TiB                 │ 30.18 MiB                │
4. │ Wide      │ Packed            │         10 │     187 │ 110.01 MiB               │ 1.30 KiB                 │
   └───────────┴───────────────────┴────────────┴─────────┴──────────────────────────┴──────────────────────────┘
```

Feb 3, 2026 · 6 min read
title_wip

ClickHouse is popular for logs and metrics analysis because of the real-time analytics capabilities provided. Ready to find out more?


## Using ClickHouse for Log Analytics[​](#using-clickhouse-for-log-analytics "Direct link to Using ClickHouse for Log Analytics")

ClickHouse is popular for logs and metrics analysis because of the real-time analytics capabilities provided.

See our official [Observability guide](/docs/use-cases/observability) for further details.

May 10, 2023 · One min read
title_wip

## How much disk space are my projection using[​](#how-much-disk-space-are-my-projection-using "Direct link to How much disk space are my projection using")

```
SELECT
    name,
    parent_name,
    formatReadableSize(bytes_on_disk) AS bytes,
    formatReadableSize(parent_bytes_on_disk) AS parent_bytes,
    bytes_on_disk / parent_bytes_on_disk AS ratio
FROM system.projection_parts
```

## Show disk storage, number of parts, number of rows in system.parts and marks across databases[​](#show-disk-storage-number-of-parts-number-of-rows-in-systemparts-and-marks-across-databases "Direct link to Show disk storage, number of parts, number of rows in system.parts and marks across databases")

```
SELECT
    database,
    table,
    partition,
    count() AS parts,
    formatReadableSize(sum(bytes_on_disk)) AS bytes_on_disk,
    formatReadableQuantity(sum(rows)) AS rows,
    sum(marks) AS marks
FROM system.parts
WHERE (database != 'system') AND active
GROUP BY
    database,
    table,
    partition
ORDER BY database ASC
```

## List details of recently written new parts[​](#list-details-of-recently-written-new-parts "Direct link to List details of recently written new parts")

The details include when they got created, how large they are, how many rows, and more:

```
SELECT
    modification_time,
    rows,
    formatReadableSize(bytes_on_disk),
    *
FROM clusterAllReplicas(default, system.parts)
WHERE (database = 'default') AND active AND (level = 0)
ORDER BY modification_time DESC
LIMIT 100
```

Mar 17, 2023 · 4 min read
title_wip

Learn how to use ClickHouse for vector search, including storing embeddings and searching with distance functions like cosine similarity.


## ClickHouse for Vector Search![​](#clickhouse-for-vector-search "Direct link to ClickHouse for Vector Search!")

Yes, ClickHouse can perform vector search.

The main advantages of using ClickHouse for vector search compared to using more specialized vector databases include:

- Using ClickHouse's filtering and full-text search capabilities to refine your dataset before performing a search.
- Performing analytics on your datasets.
- Running a `JOIN` against your existing data.
- No need to manage yet another database and complicate your infrastructure.

Here is a quick tutorial on how to use ClickHouse for vector search.

## 1. Create embeddings[​](#1-create-embeddings "Direct link to 1. Create embeddings")

Your data (documents, images, or structured data) must be converted to *embeddings*. We recommend creating embeddings using the [OpenAI Embeddings API](https://platform.openai.com/docs/api-reference/embeddings) or using the open-source Python library [SentenceTransformers](https://www.sbert.net/).

You can think of an embedding as a large array of floating-point numbers that represent your data. [Check out this guide from OpenAI to learn more about embeddings](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings).

## 2. Store the embeddings[​](#2-store-the-embeddings "Direct link to 2. Store the embeddings")

Once you have generated embeddings, you need to store them in ClickHouse. Each embedding should be stored in a separate row and can include metadata for filtering, aggregations, or analytics. Here's an example of a table that can store images with captions:

```
CREATE TABLE images
(
	`_file` LowCardinality(String),
	`caption` String,
	`image_embedding` Array(Float32)
)
ENGINE = MergeTree;
```

## 3. Search for related embeddings[​](#3-search-for-related-embeddings "Direct link to 3. Search for related embeddings")

Let's say you want to search for pictures of dogs in your dataset. You can use a distance function like `cosineDistance` to take an embedding of a dog image and search for related images:

```
SELECT
    _file,
	caption,
	cosineDistance(
        -- An embedding of your "input" dog picture
        [0.5736801028251648, 0.2516217529773712, ...,  -0.6825592517852783],
        image_embedding
    ) AS score
FROM images
ORDER BY score ASC
LIMIT 10
```

This query returns the `_file` names and `caption` of the top 10 images most likely to be related to your provided dog image.

## Further Reading[​](#further-reading "Direct link to Further Reading")

To follow a more in-depth tutorial on vector search using ClickHouse, please see:

- [Exact and Approximate Vector Search](https://clickhouse.com/docs/engines/table-engines/mergetree-family/annindexes)

Oct 26, 2023 · 2 min read
title_wip

Monitor the number of active or queued mutations in ClickHouse, especially when performing `ALTER` or `UPDATE` operations. Use the `system.mutations` table for tracking mutations.


## Question[​](#question "Direct link to Question")

How do I view the number of active or queued mutations?

## Answer[​](#answer "Direct link to Answer")

Monitoring the number of active or queued mutations is important if you are performing a lot of `ALTER` or `UPDATE` statements on your tables. These queries rewrite data parts and are not atomic - they are ordered by their creation part and applied to each part in that order. You can find more details on mutations in the [docs](https://clickhouse.com/docs/sql-reference/statements/alter#mutations).

Each mutation generates an entry in the `system.mutations` table. When performing a large number of mutations, you can monitor the count running and queued mutations with this:

```
SELECT
   hostname() AS host,
   count()
   FROM clusterAllReplicas('default', 'system.mutations') WHERE not is_done
   GROUP BY host;
```

Note

This query assumes you are running a cluster named `default`, which is the name of your cluster in ClickHouse Cloud. Replace `default` with the name of your cluster.

If you do not have a cluster, use this command:

```
SELECT
   hostname() AS host,
   count()
   FROM system.mutations WHERE not is_done
   GROUP BY host;
```

We also recommend reading this recent [blog on updates and deletes](https://clickhouse.com/blog/handling-updates-and-deletes-in-clickhouse).

Jun 7, 2023 · One min read
title_wip

Being an open-source product makes this question not so straightforward to answer. You do not have to tell anyone if you want to start using ClickHouse, you just go grab source code or pre-compiled packages.


## ClickHouse Users[​](#clickhouse-users "Direct link to ClickHouse Users")

Being an open-source product makes this question not so straightforward to answer. You do not have to tell anyone if you want to start using ClickHouse, you just go grab source code or pre-compiled packages.

There’s no contract to sign and the [Apache 2.0 license](https://github.com/ClickHouse/ClickHouse/blob/master/LICENSE) allows for unconstrained software distribution.

Also, the technology stack is often in a grey zone of what’s covered by an NDA. Some companies consider technologies they use as a competitive advantage even if they are open-source and do not allow employees to share any details publicly. Some see some PR risks and allow employees to share implementation details only with their PR department approval.

So how to tell who is using ClickHouse?

One way is to **ask around**. If it’s not in writing, people are much more willing to share what technologies are used in their companies, what the use cases are, what kind of hardware is used, data volumes, etc. We’re talking with users regularly on [ClickHouse Meetups](https://www.youtube.com/channel/UChtmrD-dsdpspr42P_PyRAw/playlists) all over the world and have heard stories about 1000+ companies that use ClickHouse. Unfortunately, that’s not reproducible and we try to treat such stories as if they were told under NDA to avoid any potential troubles. But you can come to any of our future meetups and talk with other users on your own. There are multiple ways how meetups are announced, for example, you can subscribe to [our Twitter](https://twitter.com/ClickHouseDB/).

The second way is to look for companies **publicly saying** that they use ClickHouse. It’s more substantial because there’s usually some hard evidence like a blog post, talk video recording, slide deck, etc. We collect the collection of links to such evidence on our **[Adopters](https://clickhouse.com/docs/about-us/adopters)** page. Feel free to contribute the story of your employer or just some links you’ve stumbled upon (but try not to violate your NDA in the process).

You can find names of very large companies in the adopters list, like Bloomberg, Cisco, China Telecom, Tencent, or Uber, but with the first approach, we found that there are many more. For example, if you take [the list of largest IT companies by Forbes (2020)](https://www.forbes.com/sites/hanktucker/2020/05/13/worlds-largest-technology-companies-2020-apple-stays-on-top-zoom-and-uber-debut/) over half of them are using ClickHouse in some way. Also, it would be unfair not to mention [Yandex](https://clickhouse.com/docs/about-us/history), the company which initially open-sourced ClickHouse in 2016 and happens to be one of the largest IT companies in Europe.

Sep 1, 2021 · 2 min read
title_wip

Learn why the ClickHouse developers chose to set a verbose logging level by default.


## Verbose Logging[​](#verbose-logging "Direct link to Verbose Logging")

One thing that often confuses new users is that ClickHouse generates a lot of logging output, even under light load.

This is because the default logging level, due to historical reasons, is `trace` (instead of `warning` that would be the default in other databases).

The ClickHouse developers argue that `trace` provides a lot of insight in case something goes wrong.

On the other hand, large volumes of logging mean that the system table `system.text_log` fills up quickly and needs to be merged in the background.

If the database runs stably, users may re-configure the log level, which we explain how to do below.

## Change the log level[​](#change-the-log-level "Direct link to Change the log level")

The different log levels available are documented [here](https://clickhouse.com/docs/operations/server-configuration-parameters/settings#logger)

You'll need to edit the ClickHouse server configuration file (`/etc/clickhouse-server/config.xml`) to modify the log level.

The default value is `trace` but you can change it to the desired level. See the comment below:

```
<clickhouse>
    <logger>
        <!-- Possible levels [1]:

          - none (turns off logging)
          - fatal
          - critical
          - error
          - warning
          - notice
          - information
          - debug
          - trace
          - test (not for production usage)

            [1]: https://github.com/pocoproject/poco/blob/poco-1.9.4-release/Foundation/include/Poco/Logger.h#L105-L114
        -->
        <level>trace</level>
... Rest of the configuration file
</clickhouse>
```

Jan 10, 2025 · 2 min read
title_wip

ClickHouse Keeper improves upon ZooKeeper with features like reduced disk space usage, faster recovery, and less memory consumption, offering better performance for ClickHouse clusters.



---

## Question[​](#question "Direct link to Question")

ClickHouse Keeper provides the coordination system for data replication and distributed DDL queries execution. ClickHouse Keeper is compatible with ZooKeeper, but it might not be obvious why you should use ClickHouse Keeper instead of ZooKeeper. This article discusses some of the benefits of Keeper.

## Answer[​](#answer "Direct link to Answer")

[ClickHouse Cloud](https://console.clickhouse.cloud/) uses `clickhouse-keeper` at large scale for thousands of services in a multi-tenant environment. We designed and built Keeper so that we could remove our dependency on the Java-based ZooKeeper implementation. ClickHouse Keeper solves many well-known drawbacks of ZooKeeper and makes additional improvements, including:

- Snapshots and logs consume much less disk space due to better compression
- No limit on the default packet and node data size (it is 1 MB in ZooKeeper)
- No `zxid` overflow issue (it forces a restart for every 2B transactions in ZooKeeper)
- Faster recovery after network partitions due to the use of a better distributed consensus protocol
- It uses less memory for the same volume of data
- It is easier to setup, and it does not require specifying the JVM heap size or a custom garbage collection implementation
- A few custom commands in the protocol enable faster operations in `ReplicatedMergeTree` tables
- A larger coverage by Jepsen tests

In addition, ClickHouse Support has observed a massive decrease in cluster problems in cases with sites who use `clickhouse-keeper` rather than ZooKeeper.

Check out the [Keeper docs page](https://clickhouse.com/docs/guides/sre/keeper/clickhouse-keeper) for more details on how to configure and run ClickHouse Keeper.

Jun 7, 2023 · 2 min read
Cloud IP addresses

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/cloud/guides/data_sources/01_cloud-endpoints-api.md)

## Static IPs API[​](#static-ips-api "Direct link to Static IPs API")

If you need to fetch the list of static IPs, you can use the following ClickHouse Cloud API endpoint: [`https://api.clickhouse.cloud/static-ips.json`](https://api.clickhouse.cloud/static-ips.json). This API provides the endpoints for ClickHouse Cloud services, such as ingress/egress IPs and S3 endpoints per region and cloud.

If you're using an integration like the MySQL or PostgreSQL Engine, it is possible that you need to authorize ClickHouse Cloud to access your instances. You can use this API to retrieve the public IPs and configure them in `firewalls` or `Authorized networks` in GCP or in `Security Groups` for Azure, AWS, or in any other infrastructure egress management system you're using.

For example, to allow access from a ClickHouse Cloud service hosted on AWS in the region `ap-south-1`, you can add the `egress_ips` addresses for that region:

```
❯ curl -s https://api.clickhouse.cloud/static-ips.json | jq '.'
{
  "aws": [
    {
      "egress_ips": [
        "3.110.39.68",
        "15.206.7.77",
        "3.6.83.17"
      ],
      "ingress_ips": [
        "15.206.78.111",
        "3.6.185.108",
        "43.204.6.248"
      ],
      "region": "ap-south-1",
      "s3_endpoints": "vpce-0a975c9130d07276d"
    },
...
```

For example, an AWS RDS instance running in `us-east-2` that needs to connect to a ClickHouse cloud service should have the following Inbound security group rules:

For the same ClickHouse Cloud service running in `us-east-2`, but this time connected to an MySQL in GCP, the `Authorized networks` should look like this:

- [Static IPs API](#static-ips-api)

Was this page helpful?
ClickPipes & Language Clients

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/cloud/features/02_integrations/01_clickpipes_and_language_clients.md)

ClickHouse Cloud allows you to connect the tools and services that you love.

## Managed integration pipelines for ClickHouse Cloud[​](#clickpipes "Direct link to Managed integration pipelines for ClickHouse Cloud")

ClickPipes is a managed integration platform that makes ingesting data from a diverse set of sources as simple as clicking a few buttons.
Designed for the most demanding workloads, ClickPipes's robust and scalable architecture ensures consistent performance and reliability.
ClickPipes can be used for long-term streaming needs or one-time data loading job.