ClickHouse/clickhouse-docs-embeddings huggingface dataset:
title_wip

[Edit this page](https://github.com/ClickHouse/clickhouse-docs/blob/main/docs/cloud/reference/01_changelog/02_release_notes/25_06.md)

## Backward incompatible change[​](#backward-incompatible-change "Direct link to Backward incompatible change")

- Function `geoToH3()` now accepts the input in the order (lat, lon,res) (which is the standard order for geometric functions). Users who wish to retain the legacy result order (lon, lat,res) can set setting `geotoh3_lon_lat_input_order = true`. [#78852](https://github.com/ClickHouse/ClickHouse/pull/78852) ([Pratima Patel](https://github.com/pratimapatel2008)).
- Indexes of type `full_text` were renamed to `gin`. This follows the more familiar terminology of PostgreSQL and other databases. Existing indexes of type `full_text` remain loadable but they will throw an exception (suggesting `gin` indexes instead) when one tries to use them in searches. [#79024](https://github.com/ClickHouse/ClickHouse/pull/79024) ([Robert Schulze](https://github.com/rschu1ze)).
- Add a filesystem cache setting `allow_dynamic_cache_resize`, by default `false`, to allow dynamic resize of filesystem cache. Why: in certain environments (ClickHouse Cloud) all the scaling events happen through the restart of the process and we would love this feature to be explicitly disabled to have more control over the behavior + as a safety measure. This PR is marked as backward incompatible, because in older versions dynamic cache resize worked by default without special setting. [#79148](https://github.com/ClickHouse/ClickHouse/pull/79148) ([Kseniia Sumarokova](https://github.com/kssenii)).
- Removed support for legacy index types `annoy` and `usearch`. Both have been stubs for a long time, i.e. every attempt to use the legacy indexes returned an error anyways. If you still have `annoy` and `usearch` indexes, please drop them. [#79802](https://github.com/ClickHouse/ClickHouse/pull/79802) ([Robert Schulze](https://github.com/rschu1ze)).
  #\* Remove `format_alter_commands_with_parentheses` server setting. The setting was introduced and disabled by default in 24.2. It was enabled by default in 25.2. As there are no LTS versions that don't support the new format, we can remove the setting. [#79970](https://github.com/ClickHouse/ClickHouse/pull/79970) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)).
  #\* Minor: Force backup\_threads and restore\_threads server settings to be non zero. [#80224](https://github.com/ClickHouse/ClickHouse/pull/80224) ([Raúl Marín](https://github.com/Algunenano)).
- Fix bitNot() for String to return a zero-terminated string. [#80791](https://github.com/ClickHouse/ClickHouse/pull/80791) ([Azat Khuzhin](https://github.com/azat)).

## New feature[​](#new-feature "Direct link to New feature")
title_wip

- Added `Bech32` and `Bech32m` encoding and decoding functions (issue [#40381](https://github.com/ClickHouse/ClickHouse/issues/40381)). [#80239](https://github.com/ClickHouse/ClickHouse/pull/80239) ([George Larionov](https://github.com/glarik)).
- Support `disk` setting for Atomic and Ordinary DB engines, specifying the disk to store table metadata files. [#80546](https://github.com/ClickHouse/ClickHouse/pull/80546) ([Tuan Pham Anh](https://github.com/tuanpach)).
- Support functions to unpack and compare merge tree parts. [#80573](https://github.com/ClickHouse/ClickHouse/pull/80573) ([Mikhail Artemenko](https://github.com/Michicosun)).
- `timeSeries*` helper functions to speed up some scenarios when working with time series data: - re-sample the data to the time grid with specified start timestamp, end timestamp and step - calculate PromQL-like `delta`, `rate`, `idelta` and `irate`. [#80590](https://github.com/ClickHouse/ClickHouse/pull/80590) ([Alexander Gololobov](https://github.com/davenger)).
- Allow filtering of parts selected for querying by the disk they reside on. [#80650](https://github.com/ClickHouse/ClickHouse/pull/80650) ([tanner-bruce](https://github.com/tanner-bruce)).
- Add a landing page with the list of embedded web tools. It will open when requested by a browser-like user agent. [#81129](https://github.com/ClickHouse/ClickHouse/pull/81129) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
- Allow for filtering `NULL` values in `arrayFirst`, `arrayFirstIndex`, `arrayLast` & `arrayLastIndex`. Fixes [#81113](https://github.com/ClickHouse/ClickHouse/issues/81113). [#81197](https://github.com/ClickHouse/ClickHouse/pull/81197) ([Lennard Eijsackers](https://github.com/Blokje5)).
title_wip

## Experimental feature[​](#experimental-feature "Direct link to Experimental feature")

- Hive metastore catalog for iceberg datalake. [#77677](https://github.com/ClickHouse/ClickHouse/pull/77677) ([Konstantin Vedernikov](https://github.com/scanhex12)).
- The explicit parameters are supported via key-value pairs. Currently, supported parameters are a mandatory `tokenizer` and two optional `max_rows_per_postings_list` and `ngram_size`. [#80262](https://github.com/ClickHouse/ClickHouse/pull/80262) ([Elmi Ahmadov](https://github.com/ahmadov)).
- Experimental indexes of type `gin` were renamed to `text`. Existing indexes of type `gin` remain loadable but they will throw an exception (suggesting `text` indexes instead) when one tries to use them in searches. [#80855](https://github.com/ClickHouse/ClickHouse/pull/80855) ([Robert Schulze](https://github.com/rschu1ze)).

## Performance improvement[​](#performance-improvement "Direct link to Performance improvement")
title_wip

- Improve the shutdown speed of Kafka engine (remove an extra 3 seconds delay in case of multiple Kafka tables). [#80796](https://github.com/ClickHouse/ClickHouse/pull/80796) ([Azat Khuzhin](https://github.com/azat)).
- Reduce memory usage of async inserts and improve the performance of insert queries. [#80972](https://github.com/ClickHouse/ClickHouse/pull/80972) ([Raúl Marín](https://github.com/Algunenano)).
- Don't profile processors if the log table is disabled. [#81256](https://github.com/ClickHouse/ClickHouse/pull/81256) ([Raúl Marín](https://github.com/Algunenano)).
- Speed up `toFixedString` when the source is exactly what's requested. [#81257](https://github.com/ClickHouse/ClickHouse/pull/81257) ([Raúl Marín](https://github.com/Algunenano)).
- Don't process quota values if the user is not limited. [#81549](https://github.com/ClickHouse/ClickHouse/pull/81549) ([Raúl Marín](https://github.com/Algunenano)).
- Make ProcfsMetricsProvider thread\_local to keep files open between tasks. [#81576](https://github.com/ClickHouse/ClickHouse/pull/81576) ([Raúl Marín](https://github.com/Algunenano)).
- Fixed performance regression in memory tracking. [#81694](https://github.com/ClickHouse/ClickHouse/pull/81694) ([Michael Kolupaev](https://github.com/al13n321)).
title_wip

## Improvement[​](#improvement "Direct link to Improvement")

#\* `clickhouse-local` will retain its databases after restart if you specify the `--path` command line argument. This closes [#50647](https://github.com/ClickHouse/ClickHouse/issues/50647). This closes [#49947](https://github.com/ClickHouse/ClickHouse/issues/49947). [#71722](https://github.com/ClickHouse/ClickHouse/pull/71722) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
title_wip

- [Backward incompatible change](#backward-incompatible-change)- [New feature](#new-feature)- [Experimental feature](#experimental-feature)- [Performance improvement](#performance-improvement)- [Improvement](#improvement)- [Bug fix (user-visible misbehavior in an official stable release)](#bug-fix-user-visible-misbehavior-in-an-official-stable-release)

Was this page helpful?
v26.2 Changelog for Cloud

- Fixed combination of `use_const_adaptive_granularity` and `index_granularity_bytes` (non-adaptive granularity) leading to miscalculation. [#96143](https://github.com/ClickHouse/ClickHouse/pull/96143) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
- Fixed `Number of rows in lazy chunk does not match number of offsets` exception when reading from tables with non-adaptive index granularity using `ORDER BY ... LIMIT`. [#97270](https://github.com/ClickHouse/ClickHouse/pull/97270) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
- Fixed regression in zero-copy replication where shared parts could be deleted before other replicas finished fetching them. [#95597](https://github.com/ClickHouse/ClickHouse/pull/95597) ([filimonov](https://github.com/filimonov)).
- Fixed `SYSTEM RESTART REPLICA` losing table from database when table re-creation failed with a non-ZooKeeper exception, causing metadata digest mismatches in `DatabaseReplicated`. [#97276](https://github.com/ClickHouse/ClickHouse/pull/97276) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
- Fixed `assert_cast` exception when building column statistics after `ALTER TABLE MODIFY COLUMN` changed the column type. [#97027](https://github.com/ClickHouse/ClickHouse/pull/97027) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
- When a table has no statistics defined, ClickHouse no longer tries to load them, avoiding unnecessary overhead. [#96233](https://github.com/ClickHouse/ClickHouse/pull/96233) ([Han Fei](https://github.com/hanfei1991)).
- Fixed `There was an error: Cannot obtain error message` logical error when waiting for a distributed DDL and dropping the Replicated database concurrently. [#95664](https://github.com/ClickHouse/ClickHouse/pull/95664) ([Alexander Tokmakov](https://github.com/tavplubix)).
- Fixed logical error in `KeyCondition` when a table had a nullable primary key and the query used `coalesce` with a constant first argument. [#96340](https://github.com/ClickHouse/ClickHouse/pull/96340) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
v26.2 Changelog for Cloud

- [Backward Incompatible Changes](#backward-incompatible-changes)
  - [Query and syntax changes](#query-and-syntax-changes)- [Data type changes](#data-type-changes)- [Storage and index changes](#storage-and-index-changes)- [Removed features](#removed-features)- [Settings and configuration changes](#settings-and-configuration-changes)- [Security and access control changes](#security-and-access-control-changes)- [Insert and deduplication changes](#insert-and-deduplication-changes)- [System table changes](#system-table-changes)- [Other breaking changes](#other-breaking-changes)- [New Features](#new-features)
    - [Authentication](#authentication)- [Functions](#functions)- [System tables](#system-tables)- [Table engines and storage](#table-engines-and-storage)- [Insert and deduplication](#insert-and-deduplication)- [SQL and query features](#sql-and-query-features)- [Settings and configuration](#settings-and-configuration)- [Monitoring](#monitoring)- [Experimental Features](#experimental-features)- [Improvements](#improvements)
        - [Query and SQL](#query-and-sql-improvements)- [Table engines and storage](#table-engines-and-storage-improvements)- [Data lakes](#data-lakes-improvements)- [S3Queue](#s3queue-improvements)- [S3 and object storage](#s3-and-object-storage-improvements)- [Functions](#functions-improvements)- [Settings and configuration](#settings-and-configuration-improvements)- [System tables and monitoring](#system-tables-and-monitoring-improvements)- [ClickHouse Keeper](#clickhouse-keeper-improvements)- [Memory management](#memory-management-improvements)- [Data formats](#data-formats-improvements)- [Backup and restore](#backup-and-restore-improvements)- [Named collections and dictionaries](#named-collections-and-dictionaries)- [Deduplication](#deduplication-improvements)- [Other improvements](#other-improvements)- [Performance Improvements](#performance-improvements)
          - [JOIN performance](#join-performance)- [Query optimization](#query-optimization)- [Function and aggregation performance](#function-and-aggregation-performance)- [Storage and I/O performance](#storage-and-io-performance)- [Memory optimization](#memory-optimization)- [Internal optimizations](#internal-optimizations)- [Bug Fixes](#bug-fixes)
            - [JOIN fixes](#join-fixes)- [Query and analyzer fixes](#query-and-analyzer-fixes)- [MergeTree and storage fixes](#mergetree-and-storage-fixes)- [Data type and serialization fixes](#data-type-and-serialization-fixes)- [Text index and skip index fixes](#text-index-and-skip-index-fixes)- [Data lake fixes](#data-lake-fixes)- [S3/Azure/object storage fixes](#s3-azure-object-storage-fixes)- [S3Queue fixes](#s3queue-fixes)- [Security and access control fixes](#security-and-access-control-fixes)- [Backup and restore fixes](#backup-and-restore-fixes)- [Crash and stability fixes](#crash-and-stability-fixes)- [Other bug fixes](#other-bug-fixes)

Was this page helpful?
chDB

## About chDB[​](#about-chdb "Direct link to About chDB")

- Read the full story about the birth of the chDB project on [blog](https://clickhouse.com/blog/chdb-embedded-clickhouse-rocket-engine-on-a-bicycle)
- Read about chDB and its use cases on the [Blog](https://clickhouse.com/blog/welcome-chdb-to-clickhouse)
- Take the [chDB on-demand course](https://learn.clickhouse.com/user_catalog_class/show/1901178)
- Discover chDB in your browser using [codapi examples](https://antonz.org/trying-chdb/)
- More examples see (<https://github.com/chdb-io/chdb/tree/main/examples>)

## License[​](#license "Direct link to License")

chDB is available under the Apache License, Version 2.0. See [LICENSE](https://github.com/chdb-io/chdb/blob/main/LICENSE.txt) for more information.

- [Key features](#key-features)- [DataStore: Pandas-Compatible API](#datastore)
    - [One-Line Migration](#one-line-migration)- [Performance Highlights](#performance-highlights)- [DataStore Features](#datastore-features)- [What languages are supported by chDB?](#what-languages-are-supported-by-chdb)- [How do I get started?](#how-do-i-get-started)
        - [For pandas Users](#for-pandas-users)- [DataStore API Reference](#datastore-reference)- [SQL API Guides](#sql-guides)- [An introductory video](#an-introductory-video)- [Performance benchmarks](#performance-benchmarks)- [About chDB](#about-chdb)- [License](#license)

Was this page helpful?
title_wip

```
from chdb.datastore.config import function_config

# Force engine for functions
function_config.use_chdb(*function_names)
function_config.use_pandas(*function_names)

# Set default preference
function_config.prefer_chdb()
function_config.prefer_pandas()

# Reset to default (auto)
function_config.reset()

# Check configuration
function_config.get_engine('length')  # Returns 'chdb', 'pandas', or 'auto'
```

### Per-Call Override[​](#per-call "Direct link to Per-Call Override")

Some methods support per-call engine override:

```
# Using engine parameter (where supported)
ds['result'] = ds['col'].str.upper(engine='pandas')
```

---

## Best Practices[​](#best-practices "Direct link to Best Practices")

### 1. Start with Defaults[​](#start-with-defaults "Direct link to 1. Start with Defaults")

```
# Use auto mode, let DataStore decide
config.use_auto()
```

### 2. Configure for Specific Workloads[​](#configure-for-specific-workloads "Direct link to 2. Configure for Specific Workloads")

```
# For ClickHouse-optimized string processing
function_config.use_chdb('length', 'substring', 'concat')

# For pandas-compatible string behavior
function_config.use_pandas('upper', 'lower')
```

### 3. Use Appropriate Correction Level[​](#use-appropriate-correction-level "Direct link to 3. Use Appropriate Correction Level")

```
# Development: more permissive
config.set_correction_level(CorrectionLevel.ALL)

# Production: stricter
config.set_correction_level(CorrectionLevel.HIGH)
```

### 4. Test Both Engines[​](#test-both-engines "Direct link to 4. Test Both Engines")

```
# Test with chdb
config.use_chdb()
result_chdb = process_data()

# Test with pandas
config.use_pandas()
result_pandas = process_data()

# Compare results
assert result_chdb.equals(result_pandas)
```

- [Function Engine Configuration](#function-engine)
  - [Setting Function Engines](#setting-engines)- [When to Use](#when-to-use)- [Example](#function-example)- [Overlapping Functions](#overlapping)- [chdb-Only Functions](#chdb-only)- [pandas-Only Functions](#pandas-only)- [Dtype Correction](#dtype-correction)
          - [Correction Levels](#correction-levels)- [Correction Level Details](#level-details)- [When Types Need Correction](#when-correction)- [Example](#correction-example)- [Function Configuration API](#api)
            - [function\_config Object](#function-config-object)- [Per-Call Override](#per-call)- [Best Practices](#best-practices)
              - [1. Start with Defaults](#start-with-defaults)- [2. Configure for Specific Workloads](#configure-for-specific-workloads)- [3. Use Appropriate Correction Level](#use-appropriate-correction-level)- [4. Test Both Engines](#test-both-engines)

Was this page helpful?
title_wip

### Advanced topics[​](#advanced-topics "Direct link to Advanced topics")

- [Execution Model](/docs/chdb/datastore/execution-model) - Lazy evaluation and caching
- [Class Reference](/docs/chdb/datastore/class-reference) - Complete API reference

### Configuration & debugging[​](#configuration-debugging "Direct link to Configuration & debugging")

- [Configuration](/docs/chdb/configuration) - All configuration options
- [Performance Mode](/docs/chdb/configuration/performance-mode) - SQL-first mode for maximum throughput
- [Debugging](/docs/chdb/debugging) - Explain, profiling, and logging

### Pandas user guides[​](#pandas-user-guides "Direct link to Pandas user guides")

- [Pandas Cookbook](/docs/chdb/guides/pandas-cookbook) - Common patterns
- [Key Differences](/docs/chdb/guides/pandas-differences) - Important differences from pandas
- [Performance Guide](/docs/chdb/guides/pandas-performance) - Optimization tips
- [SQL for Pandas Users](/docs/chdb/guides/pandas-to-sql) - Understanding the SQL behind pandas operations

## Quick example[​](#quick-example "Direct link to Quick example")

```
from chdb import datastore as pd

# Read data from various sources
ds = pd.read_csv("sales.csv")
# or: ds = pd.DataStore.uri("s3://bucket/sales.parquet")
# or: ds = pd.DataStore.from_mysql("mysql://user:pass@host/db/table")

# Familiar pandas operations - automatically optimized to SQL
result = (ds
    .filter(ds['amount'] > 1000)           # WHERE amount > 1000
    .groupby('region')                      # GROUP BY region
    .agg({'amount': ['sum', 'mean']})       # SUM(amount), AVG(amount)
    .sort_values('sum', ascending=False)    # ORDER BY sum DESC
    .head(10)                               # LIMIT 10
)

# View the generated SQL
print(result.to_sql())

# Execute and get results
df = result.to_df()  # Returns pandas DataFrame
```

## Next steps[​](#next-steps "Direct link to Next steps")

- **New to DataStore?** Start with the [Quickstart Guide](/docs/chdb/datastore/quickstart)
- **Coming from pandas?** Read the [Migration Guide](/docs/chdb/guides/migration-from-pandas)
- **Want to learn more?** Explore the [API Reference](/docs/chdb/datastore/class-reference)

- [Key features](#key-features)- [Architecture](#architecture)- [One-Line migration from Pandas](#migration)- [Performance comparison](#performance)- [When to use DataStore](#when-to-use)- [Feature comparison](#comparison)- [API statistics](#api-stats)- [Documentation navigation](#navigation)
                - [Getting Started](#getting-started)- [API reference](#api-reference)- [Advanced topics](#advanced-topics)- [Configuration & debugging](#configuration-debugging)- [Pandas user guides](#pandas-user-guides)- [Quick example](#quick-example)- [Next steps](#next-steps)

Was this page helpful?
DataStore Execution Model

# Bad: Multiple separate queries
sums = ds.groupby('category')['amount'].sum().to_df()
means = ds.groupby('category')['amount'].mean().to_df()
# Two queries instead of one
```

---

## Best Practices Summary[​](#best-practices "Direct link to Best Practices Summary")

1. **Chain operations before executing** - Build the full query, then trigger once
2. **Filter early** - Reduce data at the source
3. **Select only needed columns** - Column pruning improves performance
4. **Use `explain()` to understand execution** - Debug before running
5. **Let SQL handle aggregations** - ClickHouse is optimized for this
6. **Be aware of execution triggers** - Avoid accidental early execution
7. **Use caching wisely** - Understand when cache is invalidated

- [Lazy Evaluation](#lazy-evaluation)
  - [Example: Lazy vs Eager](#lazy-vs-eager)- [Benefits of Lazy Evaluation](#benefits)- [Execution Triggers](#triggers)
    - [Automatic Triggers](#automatic-triggers)- [Operations That Stay Lazy](#stay-lazy)- [Three-Phase Execution](#three-phase)
      - [Phase 1: SQL Query Building (Lazy)](#phase-1)- [Phase 2: Execution Point](#phase-2)- [Phase 3: DataFrame Operations (if any)](#phase-3)- [Viewing Execution Plans](#explain)- [Caching](#caching)
          - [How Caching Works](#how-caching)- [Cache Invalidation](#cache-invalidation)- [Manual Cache Control](#cache-control)- [Mixing SQL and Pandas Operations](#mixing)
            - [SQL-Compatible Operations](#sql-ops)- [Pandas-Only Operations](#pandas-ops)- [Hybrid Pipelines](#hybrid)- [Execution Engine Selection](#engine-selection)
              - [Auto Mode (Default)](#auto-mode)- [Force chDB Engine](#chdb-engine)- [Force pandas Engine](#pandas-engine)- [Performance Implications](#performance)
                - [Good: Filter Early](#filter-early)- [Bad: Filter Late](#filter-late)- [Good: Select Columns Early](#select-early)- [Good: Let SQL Do the Work](#sql-work)- [Best Practices Summary](#best-practices)

Was this page helpful?
title_wip

See [I/O Operations](/docs/chdb/datastore/io) for detailed documentation.

---

## Iteration[​](#iteration "Direct link to Iteration")

|  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Method Description|  |  |  |  |  |  | | --- | --- | --- | --- | --- | --- | | `items()` Iterate (column, Series)|  |  |  |  | | --- | --- | --- | --- | | `iterrows()` Iterate (index, Series)|  |  | | --- | --- | | `itertuples()` Iterate as named tuples | | | | | | | |

---

## Key Differences from Pandas[​](#differences "Direct link to Key Differences from Pandas")

### 1. Return Types[​](#return-types "Direct link to 1. Return Types")

```
# Pandas returns Series
pdf['col']  # → pd.Series

# DataStore returns ColumnExpr (lazy)
ds['col']   # → ColumnExpr
```

### 2. Lazy Execution[​](#lazy-execution "Direct link to 2. Lazy Execution")

```
# DataStore operations are lazy
result = ds.filter(ds['age'] > 25)  # Not executed yet
df = result.to_df()  # Executed here
```

### 3. No inplace Parameter[​](#no-inplace-parameter "Direct link to 3. No inplace Parameter")

```
# Pandas
df.drop(columns=['col'], inplace=True)

# DataStore (always returns new object)
ds = ds.drop(columns=['col'])
```

### 4. Comparing Results[​](#comparing-results "Direct link to 4. Comparing Results")

```
# Use to_pandas() for comparison
pd.testing.assert_frame_equal(
    ds.to_pandas(),
    expected_df
)
```

See [Key Differences](/docs/chdb/guides/pandas-differences) for complete details.

- [Compatibility Approach](#approach)- [Attributes and Properties](#attributes)- [Indexing and Selection](#indexing)- [Statistical Methods](#statistical)- [Data Manipulation](#manipulation)- [Sorting and Ranking](#sorting)- [Reshaping](#reshaping)- [Combining / Joining](#combining)- [Binary Operations](#binary)- [Comparison Operations](#comparison)- [Function Application](#application)- [Time Series](#timeseries)- [Missing Data](#missing)- [I/O Methods](#io)- [Iteration](#iteration)- [Key Differences from Pandas](#differences)
                                - [1. Return Types](#return-types)- [2. Lazy Execution](#lazy-execution)- [3. No inplace Parameter](#no-inplace-parameter)- [4. Comparing Results](#comparing-results)

Was this page helpful?