Project: ClickHouse (App Definition and Development / Database)
File: v21.5.1.6601-prestable.md

sidebar_position: 1 sidebar_label: 2022 Change comparison of integers and floating point numbers when integer is not exactly representable in the floating point data type. In new version comparison will return false as the rounding error will occur. Example: `9223372036854775808.0 != 9223372036854775808`, because the number `9223372036854775808` is not representable as floating point number exactly (and `9223372036854775808.0` is rounded to `9223372036854776000.0`). But in previous version the comparison will return as the numbers are equal, because if the floating point number `9223372036854776000.0` get converted back to UInt64, it will yield `9223372036854775808`. For the reference, the Python programming language also treats these numbers as equal. But this behaviour was dependend on CPU model (different results on AMD64 and AArch64 for some out-of-range numbers), so we make the comparison more precise. It will treat int and float numbers equal only if int is represented in floating point type exactly. (). Implement function `arrayFold(x1,...,xn,accum -> expression, array1,...,arrayn, init_accum)` that applies the expression to each element of the array (or set of parallel arrays) and collect result in accumulator. (). - Support Apple m1. (). Add a setting `maxdistributeddepth` that limits the depth of recursive queries to `Distributed` tables. Closes . (). Table function, which allows to process files from `s3` in parallel from many nodes in a specified cluster. (). Support for replicas in MySQL/PostgreSQL table engine / table function. Added wrapper storage over MySQL / PostgreSQL storages to allow shards. Closes . (). Update paths to the catboost model configs in config reloading. (). Add new setting `nonreplicateddeduplication_window` for non-replicated MergeTree inserts deduplication. (). FlatDictionary added `initialarraysize`, `maxarraysize` options. (). Added `ALTER TABLE ... FETCH PART ...` query. It's similar to `FETCH PARTITION`, but fetches only one part. (). Added `Decimal256` type support in dictionaries. Closes . (). Add function alignment for possibly better performance. (). Exclude values that does not belong to the shard from right part of IN section for distributed queries (under `optimizeskipunusedshardsrewritein`, enabled by default, since it still requires `optimizeskipunusedshards`). (). Disable compression by default when interacting with localhost (with clickhouse-client or server to server with distributed queries) via native protocol. It may improve performance of some import/export operations. This closes . (). Improve performance of reading from `ArrowStream` input format for sources other then local file (e.g. URL). (). Improve performance of `intDiv` by dynamic dispatch for AVX2. This closes . (). Support dynamic interserver credentials. (). Add clickhouse-library-bridge for library dictionary source. Closes . (). Allow publishing Kafka errors to a virtual column of Kafka engine, controlled by the `kafkahandleerror_mode` setting. (). Use nanodbc instead of Poco::ODBC. Closes . Add support for DateTime64 and Decimal for ODBC table engine. Closes . Fixed issue with cyrillic text being truncated. Closes . Added connection pools for odbc bridge. (). Speeded up reading subset of columns from File-like table engine with internal file written in column oriented data formats (Parquet, Arrow and ORC) This closes Done by @keen-wolf. (). Correctly check structure of async distributed blocks. (). Make `round` function to behave consistently on non-x86_64 platforms. Rounding half to nearest even (Banker's rounding) is used.

Clear the rest of the screen and show cursor in `clickhouse-client` if previous program has left garbage in terminal. This closes . (). Allow to use CTE in VIEW definition. This closes . (). Add metric to track how much time is spend during waiting for Buffer layer lock. (). Allow RBAC row policy via postgresql protocol. Closes . PostgreSQL protocol is enabled in configuration by default. (). MaterializeMySQL (experimental feature). Make Clickhouse to be able to replicate MySQL databases containing views without failing. This is accomplished by ignoring the views. ... (). `dateDiff` now works with `DateTime64` arguments (even for values outside of `DateTime` range) ... (). Set `backgroundfetchespool_size` to 8 that is better for production usage with frequent small insertions or slow ZooKeeper cluster. (). Fix inactivepartstothrowinsert=0 with inactivepartstodelayinsert>0. (). Respect maxpartremoval_threads for ReplicatedMergeTree. (). Fix an error handling in Poco HTTP Client for AWS. (). When selecting from MergeTree table with NULL in WHERE condition, in rare cases, exception was thrown. This closes . (). Add ability to flush buffer only in background for StorageBuffer. (). Add ability to run clickhouse-keeper with SSL. Config settings `keeperserver.tcpportsecure` can be used for secure interaction between client and keeper-server. `keeperserver.raft_configuration.secure` can be used to enable internal secure communication between nodes. (). Increase `maxurisize` (the maximum size of URL in HTTP interface) to 1 MiB by default. This closes . (). Do not perform optimizeskipunused_shards for cluster with one node. (). Raised the threshold on max number of matches in result of the function `extractAllGroupsHorizontal`. (). Implement functions `arrayHasAny`, `arrayHasAll`, `has`, `indexOf`, `countEqual` for generic case when types of array elements are different. In previous versions the functions `arrayHasAny`, `arrayHasAll` returned false and `has`, `indexOf`, `countEqual` thrown exception. Also add support for `Decimal` and big integer types in functions `has` and similar. This closes . (). Fix memory tracking with minbytestousemmap_io. (). Make function `unhex` case insensitive for compatibility with MySQL. (). Fix very rare bug when quorum insert with `quorum_parallel=1` is not really "quorum" because of deduplication. (). Fix "unknown column" error for tables with `Merge` engine in queris with `JOIN` and aggregation. Closes , close . (). Check if table function view is used as a column. This complements https://github.com/ClickHouse/ClickHouse/pull/20350. (). Fix bug, which leads to underaggregation of data in case of enabled `optimizeaggregationinorder` and many parts in table. Slightly improve performance of aggregation with enabled `optimizeaggregationinorder`. (). Do not limit HTTP chunk size. Fixes . (). Buffer overflow (on read) was possible in `tokenbf_v1` full text index. The excessive bytes are not used but the read operation may lead to crash in rare cases. This closes . (). Fix ClickHouseDictionarySource configuration loop. Closes . (). Fix bug in partial merge join with `LowCardinality`. Close , close . (). Follow-up fix for . Also fixes . (). Fix deserialization of empty string without newline at end of TSV format. This closes . Possible workaround without version update: set `inputformatnullasdefault` to zero. It was zero in old versions. (). Fix UB by unlocking the rwlock of the TinyLog from the same thread.

Avoid UB in Log engines for rwlock unlock due to unlock from another thread. (). Fix usage of function `map` in distributed queries. (). Try flush write buffer only if it is initialized. Fixes segfault when client closes connection very early . (). Fixed a bug with unlimited wait for auxiliary AWS requests. (). Fix LOGICAL_ERROR for Log with nested types w/o columns in the SELECT clause. (). Fix wait for mutations on several replicas for ReplicatedMergeTree table engines. Previously, mutation/alter query may finish before mutation actually executed on other replicas. (). Fix possible hangs in zk requests in case of OOM exception. Fixes . (). Fix approx total rows accounting for reverse reading from MergeTree. (). Revert "Move conditions from JOIN ON to WHERE" (ClickHouse/ClickHouse), close , close . (). Fix pushdown of `HAVING` in case, when filter column is used in aggregation. (). LIVE VIEW (experimental feature). Fix possible hanging in concurrent DROP/CREATE of TEMPORARY LIVE VIEW in `TemporaryLiveViewCleaner`, see https://gist.github.com/vzakaznikov/0c03195960fc86b56bfe2bc73a90019e. (). Fix bytesallocated for sparsehashed dictionaries. (). Fixed a crash when using `mannWhitneyUTest` and `rankCorr` with window functions. This fixes . (). fixed `formatDateTime()` on `DateTime64` and "%C" format specifier fixed `toDateTime64()` for large values and non-zero scale. ... (). Fix usage of constant columns of type `Map` with nullable values. (). Simplify debian packages. This fixes . (). Fix error `Cannot find column in ActionsDAG result` which may happen if subquery uses `untuple`. Fixes . (). Remove non-essential details from suggestions in clickhouse-client. This closes . (). Fixed `Table .inner_id... doesn't exist` error when selecting from Materialized View after detaching it from Atomic database and attaching back. (). Some values were formatted with alignment in center in table cells in `Markdown` format. Not anymore. (). Server might fail to start if `datatypedefault_nullable` setting is enabled in default profile, it's fixed. Fixes . (). Fix missing whitespace in some exception messages about `LowCardinality` type. (). Fixed missing semicolon in exception message. The user may find this exception message unpleasant to read. (). Fix reading from ODBC when there are many long column names in a table. Closes . (). Add on-demand check for clickhouse Keeper. (). Disable incompatible libraries (platform specific typically) on ppc64le ... (). Allow building with unbundled xz (lzma) using USEINTERNALXZ_LIBRARY=OFF ... (). Allow query profiling only on x86_64. See #issuecomment-812954965 and #issuecomment-703805337. This closes . (). Adjust some tests to output identical results on amd64 and aarch64 (qemu). The result was depending on implementation specific CPU behaviour. (). Fix some tests on AArch64 platform. (). Fix ClickHouse Keeper build for MacOS. (). Fix some points from this comment https://github.com/ClickHouse/ClickHouse/pull/19516#issuecomment-782047840. (). Build `jemalloc` with support for . (). NO CL ENTRY: 'Error message reads better'. (). Fix SIGSEGV by waiting servers thread pool (). Better filter push down (). Better tests for finalize in nested writers (). Replace all Context references with std::weak_ptr (). make some perf test queries more stable (). fix ccache broken by prlimit (). Remove old MSan suppressions (part 5) (). Add test for copier (). fix window frame offset check and add more tests (). FormatSettings nullasdefault default value fix

Minor fixes in tests for AArch64 (). Filter removed/renamed tests from ci-changed-files.txt for fuzzer (). Try fix flaky test (). AppleClang compilation fix (). Fix assert in Arena when doing GROUP BY Array of Nothing of non-zero size. (). Minor improvement in index deserialization (). Fix flaky test after (). Fix comments (). Fix some uncaught exceptions (in SCOPE_EXIT) under memory pressure (). Introduce IStorage::distributedWrite method for distributed INSERT SELECTS (). Fix flaky test 00816longconcurrentaltercolumn (). FlatDictionary fix perf test (). DirectDictionary dictGet multiple columns optimization (). Better retries on ZK errors in sh tests (). Add log_comment setting for DROP/CREATE DATABASE in clickhouse-test (). Add retires for docker-compose pull in integration tests (). Fix impossible invalid-read for system.errors accounting (). Skip compiling xz if we're using system xz (unbundled) (). Fix test 01294createsettings_profile (). Fix test 01039rowpolicy_dcl (). Another attempt to enable pytest (). Fix random failures of tests that are using query_log (). fix build error 'always_inline' function might not be inlinable (). Add bool type in postgres engine (). Fix mutation killers tests (). Change Aggregatingmergetree to AggregatingMergeTree in docs (). fix window functions with multiple input streams and no sorting (). Remove redundant fsync on coordination logs rotation (). Fix test 01702systemquery_log (). MemoryStorage sync comments and code (). Fix potential segfault on Keeper startup (). Avoid using harmful function rand() (). Fix flacky hedged tests (). add more messages when flushing the logs (). Moved BorrowedObjectPool to common (). Functions ExternalDictionaries standardize exception throw (). FileDictionarySource fix absolute file path (). Small change in replicated database tests run (). Slightly improve logging messages for Distributed async sends (). Fix what looks like a trivial mistake (). Add a test for already fixed issue (). DataTypeLowCardinality format tsv parsing issue (). Updated MariaDB connector fix cmake (). Prettify logs of integration tests (). Util `memcpy-bench` is built only when position independent code is disabled (). Fix vanilla GCC compilation in macOS (). Better diagnostics for OOM in stress test (). Dictionaries updated performance tests (). IAggreagteFunction allocatesMemoryInArena removed default implementation (). Fix flapping testmergetree_s3 test (). less flaky test (). Dictionaries standardize exceptions (). StorageExternalDistributed arcadia fix (). Check out of range values in FieldVisitorConverToNumber (). Fix combinators with common prefix name (State and SimpleState) with libstdc++ (). Fix arcadia (). Fix excessive warning in StorageDistributed with cross-replication (). Update MergeTreeData.cpp Better error message. (). Fix assertion when filtering tables in StorageMerge (). Add a test for (). Fix multi response in TestKeeper (). Improve hung check in Stress tests (). blog article about code review (). LibraryDictionary bridge library interface (). Remove useless files (). Upload keeper logs from stateless tests (). CI runner intergation tests logs update to tar.gz (). Tiny logging improvements (). Block all memory tracking limits in dtors/SCOPE_EXIT_SAFE/tryLogCurrentException (). jemalloc tuning (). Rename strange tests (). Fix arcadia build S3 (). Updated zlib (). (). More verbose logs for debuging test failures with Replicated and Keeper (). Fix exception message for "partstothrow_insert" (). Fix flapping tests tests3zerocopyreplication (). Add a test for (). Add test for fixed issue (). Disable postgresql_port in perf tests ().