v25.8 Changelog for Cloud

### S3 and object storage[​](#s3-and-object-storage "Direct link to S3 and object storage")

- Implement methods `moveFile` and `replaceFile` in `s3_plain_rewritable` to support it as a database disk. [#79424](https://github.com/ClickHouse/ClickHouse/pull/79424) ([Tuan Pham Anh](https://github.com/tuanpach)).
- S3 read and write requests are throttled on the HTTP socket level (instead of whole S3 requests) to avoid issues with `max_remote_read_network_bandwidth_for_server` and `max_remote_write_network_bandwidth_for_server` throttling. [#81837](https://github.com/ClickHouse/ClickHouse/pull/81837) ([Sergei Trifonov](https://github.com/serxa)).
- This PR introduces jitter to the S3 retry mechanism when the `s3_slow_all_threads_after_network_error` configuration is enabled. [#81849](https://github.com/ClickHouse/ClickHouse/pull/81849) ([zoomxi](https://github.com/zoomxi)).
- Implement AWS S3 authentication with an explicitly provided IAM role. Implement OAuth for GCS. These features were recently only available in ClickHouse Cloud and are now open-sourced. Synchronize some interfaces such as serialization of the connection parameters for object storages. [#84011](https://github.com/ClickHouse/ClickHouse/pull/84011) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
- Allow to use any storage policy (i.e. object storage, such as S3) for external aggregation/sorting. [#84734](https://github.com/ClickHouse/ClickHouse/pull/84734) ([Azat Khuzhin](https://github.com/azat)).
- Collect all removed objects to execute single object storage remove operation. [#85316](https://github.com/ClickHouse/ClickHouse/pull/85316) ([Mikhail Artemenko](https://github.com/Michicosun)).

### S3Queue table engine[​](#s3queue-table-engine "Direct link to S3Queue table engine")

- Macros like `{uuid}` can now be used in the `keeper_path` setting of the S3Queue table engine. [#82463](https://github.com/ClickHouse/ClickHouse/pull/82463) ([Nikolay Degterinsky](https://github.com/evillique)).
- Add a new server setting `s3queue_disable_streaming` which disables streaming in tables with S3Queue table engine. This setting is changeable without server restart. [#82515](https://github.com/ClickHouse/ClickHouse/pull/82515) ([Kseniia Sumarokova](https://github.com/kssenii)).
- Add columns `commit_time`, `commit_id` to `system.s3queue_log`. [#83016](https://github.com/ClickHouse/ClickHouse/pull/83016) ([Kseniia Sumarokova](https://github.com/kssenii)).
- Add logs for s3queue shutdown process. [#83163](https://github.com/ClickHouse/ClickHouse/pull/83163) ([Kseniia Sumarokova](https://github.com/kssenii)).
- Shutdown S3(Azure/etc)Queue streaming before shutting down any tables on server shutdown. [#83530](https://github.com/ClickHouse/ClickHouse/pull/83530) ([Kseniia Sumarokova](https://github.com/kssenii)).
- Support changing mv insert settings on `S3Queue` table level. Added new `S3Queue` level settings: `min_insert_block_size_rows_for_materialized_views` and `min_insert_block_size_bytes_for_materialized_views`. By default profile level settings will be used and `S3Queue` level settings will override those. [#83971](https://github.com/ClickHouse/ClickHouse/pull/83971) ([Kseniia Sumarokova](https://github.com/kssenii)).
- S3Queue ordered mode fix: quit earlier if shutdown was called. [#84463](https://github.com/ClickHouse/ClickHouse/pull/84463) ([Kseniia Sumarokova](https://github.com/kssenii)).

### Kafka integration[​](#kafka-integration "Direct link to Kafka integration")

- Count consumed messages manually to avoid depending on previous committed offset in StorageKafka2. [#81662](https://github.com/ClickHouse/ClickHouse/pull/81662) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)).
- Integrate `StorageKafka2` to [`system.kafka_consumers`](/docs/operations/system-tables/kafka_consumers). [#82652](https://github.com/ClickHouse/ClickHouse/pull/82652) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)).

### ClickHouse Keeper improvements[​](#clickhouse-keeper-improvements "Direct link to ClickHouse Keeper improvements")

- Keeper improvement: move changelog files between disk in a background thread. Previously, moving changelog to a different disk would block Keeper globally until the move is finished. This lead to performance degradation if moving is a long operation (e.g. to S3 disk). [#82485](https://github.com/ClickHouse/ClickHouse/pull/82485) ([Antonio Andelic](https://github.com/antonio2368)).
- Keeper improvement: add new config `keeper_server.cleanup_old_and_ignore_new_acl`. If enabled, all nodes will have their ACLs cleared while ACL for new requests will be ignored. If the goal is to completely remove ACL from nodes, it's important to leave the config enabled until a new snapshot is created. [#82496](https://github.com/ClickHouse/ClickHouse/pull/82496) ([Antonio Andelic](https://github.com/antonio2368)).
- Keeper improvement: support specific permissions for world:anyone ACL. [#82755](https://github.com/ClickHouse/ClickHouse/pull/82755) ([Antonio Andelic](https://github.com/antonio2368)).
- Add support for specifying extra Keeper ACL for paths in config. If you want to add extra ACL for a specific path you define it in the config under `zookeeper.path_acls`. [#82898](https://github.com/ClickHouse/ClickHouse/pull/82898) ([Antonio Andelic](https://github.com/antonio2368)).
- Adds ProfileEvent when Keeper rejects a write due to soft memory limit. [#82963](https://github.com/ClickHouse/ClickHouse/pull/82963) ([Xander Garbett](https://github.com/Garbett1)).
- Enable `create_if_not_exists`, `check_not_exists`, `remove_recursive` feature flags in Keeper by default which enable new types of requests. [#83488](https://github.com/ClickHouse/ClickHouse/pull/83488) ([Antonio Andelic](https://github.com/antonio2368)).
- Add support for applying extra ACL on specific Keeper nodes using `apply_to_children` config. [#84137](https://github.com/ClickHouse/ClickHouse/pull/84137) ([Antonio Andelic](https://github.com/antonio2368)).
- Add `get_acl` command to KeeperClient. [#84641](https://github.com/ClickHouse/ClickHouse/pull/84641) ([Antonio Andelic](https://github.com/antonio2368)).
- Add 4LW in Keeper, `lgrq`, for toggling request logging of received requests. [#84719](https://github.com/ClickHouse/ClickHouse/pull/84719) ([Antonio Andelic](https://github.com/antonio2368)).
- Reduce contention on storage lock in Keeper. [#84732](https://github.com/ClickHouse/ClickHouse/pull/84732) ([Antonio Andelic](https://github.com/antonio2368)).
- The `encrypt_decrypt` tool now supports encrypted ZooKeeper connections. [#84764](https://github.com/ClickHouse/ClickHouse/pull/84764) ([Roman Vasin](https://github.com/rvasin)).
- Limit Keeper log entry cache size by number of entries using `keeper_server.coordination_settings.latest_logs_cache_entry_count_threshold` and `keeper_server.coordination_settings.commit_logs_cache_entry_count_threshold`. [#84877](https://github.com/ClickHouse/ClickHouse/pull/84877) ([Antonio Andelic](https://github.com/antonio2368)).

### JSON and Dynamic types[​](#json-and-dynamic-types "Direct link to JSON and Dynamic types")