v25.4 Changelog for Cloud

- Optimize performance with lazy projection to avoid reading unused columns. [#55518](https://github.com/ClickHouse/ClickHouse/pull/55518) ([Xiaozhe Yu](https://github.com/wudidapaopao)).
- Start to compare rows from most likely unequal columns first. [#63780](https://github.com/ClickHouse/ClickHouse/pull/63780) ([UnamedRus](https://github.com/UnamedRus)).
- Optimize RowBinary input format. Closes [#63805](https://github.com/ClickHouse/ClickHouse/issues/63805). [#65059](https://github.com/ClickHouse/ClickHouse/pull/65059) ([Pavel Kruglov](https://github.com/Avogar)).
- Speedup string deserialization by some low-level optimisation. [#65948](https://github.com/ClickHouse/ClickHouse/pull/65948) ([Nikita Taranov](https://github.com/nickitat)).
- Apply `preserve_most` attribute at some places in code. [#67778](https://github.com/ClickHouse/ClickHouse/pull/67778) ([Nikita Taranov](https://github.com/nickitat)).
- Implement query condition cache to improve query performance using repeated conditions. The range of the portion of data that does not meet the condition is remembered as a temporary index in memory. Subsequent queries will use this index. close [#67768](https://github.com/ClickHouse/ClickHouse/issues/67768) ### Documentation entry for user-facing changes. [#69236](https://github.com/ClickHouse/ClickHouse/pull/69236) ([zhongyuankai](https://github.com/zhongyuankai)).
- Support async io prefetch for `NativeORCBlockInputFormat`, which improves overall performance by hiding remote io latency. Speedup ratio could reach 1.47x in my test case. [#70534](https://github.com/ClickHouse/ClickHouse/pull/70534) ([李扬](https://github.com/taiyang-li)).
- Improve grace hash join performance by rerange the right join table by keys. [#72237](https://github.com/ClickHouse/ClickHouse/pull/72237) ([kevinyhzou](https://github.com/KevinyhZou)).
- Reintroduce respect `ttl_only_drop_parts` on `materialize ttl`; only read the necessary columns to recalculate TTL and drop parts by replacing them with an empty one. [#72751](https://github.com/ClickHouse/ClickHouse/pull/72751) ([Andrey Zvonov](https://github.com/zvonand)).
- Allow `arrayROCAUC` and `arrayAUCPR` to compute partial area of the whole curve, so that its calculation can be parallelized over huge datasets. [#72904](https://github.com/ClickHouse/ClickHouse/pull/72904) ([Emmanuel](https://github.com/emmanuelsdias)).
- Avoid spawn too many idle threads. [#72920](https://github.com/ClickHouse/ClickHouse/pull/72920) ([Guo Wangyang](https://github.com/guowangy)).
- Splitting of left table blocks by hash was removed from the probe phase of the `parallel_hash` JOIN algorithm. [#73089](https://github.com/ClickHouse/ClickHouse/pull/73089) ([Nikita Taranov](https://github.com/nickitat)).
- Don't list blob storage keys if we only have curly brackets expansion in table function. Closes [#73333](https://github.com/ClickHouse/ClickHouse/issues/73333). [#73518](https://github.com/ClickHouse/ClickHouse/pull/73518) ([Konstantin Bogdanov](https://github.com/thevar1able)).
- Replace Int256 and UInt256 with clang builtin i256 in arithmetic calculation according to tests in [#70502](https://github.com/ClickHouse/ClickHouse/issues/70502). [#73658](https://github.com/ClickHouse/ClickHouse/pull/73658) ([李扬](https://github.com/taiyang-li)).
- Adds a fast path for functions with all argument types is numeric. Fix performance issues in <https://github.com/ClickHouse/ClickHouse/pull/72258>. [#73820](https://github.com/ClickHouse/ClickHouse/pull/73820) ([李扬](https://github.com/taiyang-li)).
- Do not apply `maskedExecute` on non-function columns, improve the performance of short circuit execution. [#73965](https://github.com/ClickHouse/ClickHouse/pull/73965) ([lgbo](https://github.com/lgbo-ustc)).
- Disable header detection for Kafka/NATS/RabbitMQ/FileLog to improve performance. [#74006](https://github.com/ClickHouse/ClickHouse/pull/74006) ([Azat Khuzhin](https://github.com/azat)).
- Use log wrappers by value and don't allocate them in a heap. [#74034](https://github.com/ClickHouse/ClickHouse/pull/74034) ([Mikhail Artemenko](https://github.com/Michicosun)).
- Execute a pipeline with a higher degree of parallelism after aggregation with grouping sets. [#74082](https://github.com/ClickHouse/ClickHouse/pull/74082) ([Nikita Taranov](https://github.com/nickitat)).
- Reduce critical section in `MergeTreeReadPool`. [#74202](https://github.com/ClickHouse/ClickHouse/pull/74202) ([Guo Wangyang](https://github.com/guowangy)).
- Optimized function `indexHint`. Now, columns that are only used as arguments of function `indexHint` are not read from the table. [#74314](https://github.com/ClickHouse/ClickHouse/pull/74314) ([Anton Popov](https://github.com/CurtizJ)).
- Parallel replicas performance improvement. Packets deserialization on query initiator, for packets not related to parallel replicas protocol, now always happens in pipeline thread. Before, it could happen in a thread responsible for pipeline scheduling, which could make initiator less responsive and delay pipeline execution. [#74398](https://github.com/ClickHouse/ClickHouse/pull/74398) ([Igor Nikonov](https://github.com/devcrafter)).
- Fixed calculation of size in memory for `LowCardinality` columns. [#74688](https://github.com/ClickHouse/ClickHouse/pull/74688) ([Nikita Taranov](https://github.com/nickitat)).
- Improves the performance of whole JSON column reading in Wide parts from S3. It's done by adding prefetches for sub column prefixes deserialization, cache of deserialized prefixes and parallel deserialization of subcolumn prefixes. It improves reading of the JSON column from S3 4 times in query like `SELECT data FROM table` and about 10 times in query like `SELECT data FROM table LIMIT 10`. [#74827](https://github.com/ClickHouse/ClickHouse/pull/74827) ([Pavel Kruglov](https://github.com/Avogar)).
- Preallocate memory used by async inserts to improve performance. [#74945](https://github.com/ClickHouse/ClickHouse/pull/74945) ([Ilya Golshtein](https://github.com/ilejn)).
- Fixed double pre-allocation in `ConcurrentHashJoin` in case join sides are swapped by the optimizer. [#75149](https://github.com/ClickHouse/ClickHouse/pull/75149) ([Nikita Taranov](https://github.com/nickitat)).
- Fixed unnecessary contention in `parallel_hash` when `max_rows_in_join = max_bytes_in_join = 0`. [#75155](https://github.com/ClickHouse/ClickHouse/pull/75155) ([Nikita Taranov](https://github.com/nickitat)).
- Slight improvement in some join scenarios: precalculate number of output rows and reserve memory for them. [#75376](https://github.com/ClickHouse/ClickHouse/pull/75376) ([Alexander Gololobov](https://github.com/davenger)).
- `plain_rewritable` metadata files are small and do not need a large default buffer. Use a write buffer sized appropriately to fit the given path, improving memory utilization for a large number of active parts. ### Documentation entry for user-facing changes. [#75758](https://github.com/ClickHouse/ClickHouse/pull/75758) ([Julia Kartseva](https://github.com/jkartseva)).
- In some cases (e.g., empty array column) data parts can contain empty files. We can skip writing empty blobs to ObjectStorage and only store metadata for such files when the table resides on disk with separated metadata and object storages. [#75860](https://github.com/ClickHouse/ClickHouse/pull/75860) ([Alexander Gololobov](https://github.com/davenger)).
- It was discovered that concurrency control could lead to unfair CPU distribution between INSERTs and SELECTs. When all CPU slots are allocated unconditionally (w/o competition) to INSERTs with `max_threads` = 1 while SELECTs with high `max_threads` values suffer from poor performance due to using only a single thread. [#75941](https://github.com/ClickHouse/ClickHouse/pull/75941) ([Sergei Trifonov](https://github.com/serxa)).
- Trivial opt on wrapInNullable to avoid unnecessary null map allocation. [#76489](https://github.com/ClickHouse/ClickHouse/pull/76489) ([李扬](https://github.com/taiyang-li)).