Project: ClickHouse (App Definition and Development / Database)
File: v20.7.1.4310-prestable.md

Deprecate special printing of zero Date/DateTime values as `0000-00-00` and `0000-00-00 00:00:00`. (). Function `modulo` (operator `%`) with at least one floating point number as argument will calculate remainder of division directly on floating point numbers without converting both arguments to integers. It makes behaviour compatible with most of DBMS. This also applicable for Date and DateTime data types. Added alias `mod`. This closes . (). - Added support of LDAP authentication for preconfigured users ("Simple Bind" method). (). Add mapAdd and mapSubtract functions for working with tuple maps. (). Added `system.crash_log` table into which stack traces for fatal errors are collected. (). Add `bayesAB` function for bayesian-ab-testing. (). Integration with . (). - Add FROM_UNIXTIME function, related to . (). A function `formatRow` is added to support turning arbitrary expressions into a string via given format. It's useful for manipulating SQL outputs and is quite versatile combined with the `columns` function. (). Add setting `allownonmetadata_alters` which restricts to execute `ALTER` queries which modify data on disk. Disabled be default. Closes . (). Add `minMap` and `maxMap` functions support to `SimpleAggregateFunction`. (). Added http headers `X-ClickHouse-Database` and `X-ClickHouse-Format` which may be used to set default database and output format. (). Implement user-defined settings. (). Optimize PK lookup for queries that match exact PK range. (). Attempt to implement streaming optimization in `DiskS3`. (). Replaces monotonous functions with its argument in ORDER BY if `set optimizemonotonousfunctionsinorder_by=1`. (). Converts String-type arguments of function "if" and "transform" into enum if `set optimizeiftransformstringsto_enum = 1`. (). Parallel PK lookup and skipping index stages on parts, as described in . (). Push down `LIMIT` step for query plan. (). Slightly improve performance of aggregation by UInt8/UInt16 keys. (). Slightly improve performance of aggregation by UInt8/UInt16 keys. (). Adds a new type of polygon dictionary which uses a recursively built grid to reduce the number of polygons which need to be checked for each point. (). Allow TabSeparatedRaw as an input format. (). Separated `AWSAuthV4Signer` into different logger, removed "AWSClient: AWSClient". (). - Implement `RENAME DATABASE` and `RENAME DICTIONARY` for `Atomic` database engine - Add implicit `{uuid}` macro, which can be used in ZooKeeper path for `ReplicatedMergeTree`. It works with `CREATE ... ON CLUSTER ...` queries. Set `showtableuuidintablecreatequeryifnot_nil` to `true` to use it. - Make `ReplicatedMergeTree` engine arguments optional, `/clickhouse/tables/{uuid}/{shard}/` and `{replica}` are used by default. Closes . - Minor fixes. - These changes break backward compatibility of `Atomic` database engine. Previously created `Atomic` databases must be manually converted to new format. (). Now joinGet supports multi-key lookup. (). Rollback insertion errors in `Log`, `TinyLog`, `StripeLog` engines. In previous versions insertion error lead to inconsisent table state (this works as documented and it is normal for these table engines). This fixes . (). Avoid overflow in parsing of DateTime values that will lead to negative unix timestamp in their timezone (for example, `1970-01-01 00:00:00` in Moscow). Saturate to zero instead. This fixes . This fixes . (). Add several metrics for requests to S3 storages. (). Changes default value for `multiplejoinsrewriter_version` to 2. It enables new multiple joins rewriter that knows about column names. (). Allow to set JOIN kind and type in more standad way: `LEFT SEMI JOIN` instead of `SEMI LEFT JOIN`. For now both are correct. (). Fix explain query format overwrite by default, issue . (). Add SelectedRows and SelectedBytes events. (). Add `currentdatabase` information to `system.querylog`. (). Support truncate table without table keyword.

Now exceptions forwarded to the client if an error happened during ALTER or mutation. Closes . (). Protect from the cases when user may set `backgroundpoolsize` to value lower than `numberoffreeentriesinpooltoexecutemutation` or `numberoffreeentriesinpooltolowermaxsizeof_merge`. In these cases ALTERs won't work or the maximum size of merge will be too limited. Saturate values instead. This closes . (). This change makes skipping index efficiency more obvious by showing both skipped and total examined granules. (). This change makes skipping index efficiency more obvious by showing total marks before and after skipping indices. (). Introduce setting `alterpartitionverbose_result` which outputs information about touched parts for some types of `ALTER TABLE ... PARTITION ...` queries (currently `ATTACH` and `FREEZE`). Closes . (). Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events. (). break-ing out of a loop because it makes sense to do so. (). Keep less amount of logs in ZooKeeper. Avoid too large growth of ZooKeeper nodes in case of offline replicas when having many servers/tables/inserts. (). Add sanity check for MergeTree settings. If the settings are incorrect, the server will refuse to start or to create a table, printing detailed explanation to the user. (). Allow Map aggregate functions to work on Arrays with NULLs. Fixes . (). Fix assert in parseDateTimeBestEffort. This fixes . (). Fix assert in geohashesInBox. This fixes . (). Now broken parts are also reported when encountered in compact part processing. (). Fix a 'Week'-interval formatting for ATTACH/ALTER/CREATE QUOTA-statements. (). Updated gitignore-files. (). kafka: fix SIGSEGV if there is an message with error in the middle of the batch. (). If MergeTree table does not contain ORDER BY or PARTITION BY, it was possible to request ALTER to CLEAR all the columns and ALTER will stuck. Fixed . (). SystemLog: do not write to ordinary server log under mutex. This can lead to deadlock if `text_log` is enabled. (). Fix crash in JOIN with dictionary when we are joining over expression of dictionary key: `t JOIN dict ON expr(dict.id) = t.id`. Disable dictionary join optimisation for this case. (). Fix SETTINGS parse after FORMAT. (). Fix backwards compatibility in binary format of `AggregateFunction(avg, ...)` values. This fixes . (). Fixed performance issue, while reading from compact parts. (). Fixing race condition in live view tables which could cause data duplication. (). Better exception for function `in` with invalid number of arguments. (). Fix bug which lead to broken old parts after `ALTER DELETE` query when `enablemixedgranularity_parts=1`. Fixes . (). Now ClickHouse will recalculate checksums for parts when file `checksums.txt` is absent. Broken since . (). Remove data for Distributed tables (blocks from async INSERTs) on DROP TABLE. (). Fix race condition in external dictionaries with cache layout which can lead server crash. (). Fix lack of aliases with function `any`. (). Fix error `Output of TreeExecutor is not sorted` for `OPTIMIZE DEDUPLICATE`. Fixes . (). Fix possible `Pipeline stuck` error for queries with external sorting. Fixes . (). The function `groupArrayMoving` was not working for distributed queries. It's result was calculated within incorrect data type (without promotion to the largest type). The function `groupArrayMovingAvg` was returning integer number that was inconsistent with the `avg` function. This fixes . (). Fix error message about adaptive granularity. (). Better exception message in disk access storage. (). Exception `There is no supertype...` can be thrown during `ALTER ... UPDATE` in unexpected cases (e.g. when subtracting from UInt64 column). This fixes . This fixes . (). CREATE USER IF NOT EXISTS now doesn't throw exception if the user

This fixes . (). Add support for function `if` with `Array(UUID)` arguments. This fixes . (). Fix SIGSEGV in StorageKafka when broker is unavailable (and not only). (). fixes fix bloom filter index with const expression. (). fixes allow push predicate when subquery contains with clause. (). Fix optimization `optimizemovefunctionsoutof_any=1` in case of `any(func(<lambda>))`. (). Fix memory tracking for inputformatparallel_parsing (by attaching thread to group). (). Fix performance with large tuples, which are interpreted as functions in `IN` section. The case when user write `WHERE x IN tuple(1, 2, ...)` instead of `WHERE x IN (1, 2, ...)` for some obscure reason. (). Fix CAST(Nullable(String), Enum()). (). Fix rare bug when `ALTER DELETE` and `ALTER MODIFY COLUMN` queries executed simultaneously as a single mutation. Bug leads to an incorrect amount of rows in `count.txt` and as a consequence incorrect data in part. Also, fix a small bug with simultaneous `ALTER RENAME COLUMN` and `ALTER ADD COLUMN`. (). Fix unnecessary limiting for the number of threads for selects from local replica. (). Fix columns duplication for range hashed dictionary created from DDL query. This fixes . (). Corrected mergewithttl_timeout logic which did not work well when expiration affected more than one partition over one time interval. (Authored by @excitoon). (). Fix `Block structure mismatch` error for queries with `UNION` and `JOIN`. Fixes . (). Fix crash which was possible for queries with `ORDER BY` tuple and small `LIMIT`. Fixes . (). Add logging when the mutation is not running because of limited disk space or free threads in the background pool. (). Fix error `Cannot convert column because it is constant but values of constants are different in source and result` for remote queries which use deterministic functions in scope of query, but not deterministic between queries, like `now()`, `now64()`, `randConstant()`. Fixes . (). Fix wrong index analysis with functions. It could lead to pruning wrong parts, while reading from `MergeTree` tables. Fixes . Fixes . (). Fix segfault when mutation killed and the server tries to send the exception to the client. (). AvroConfluent: Skip Kafka tombstone records AvroConfluent: Support skipping broken records ... (). Fix DateTime64 conversion functions with constant argument. (). Fix assert in `arrayElement` function in case of array elements are Nullable and array subscript is also Nullable. This fixes . (). Fix function if with nullable constexpr as cond that is not literal NULL. Fixes . (). Return passed number for numbers with MSB set in roundUpToPowerOfTwoOrZero(). (). Fix assertion in KeyCondition when primary key contains expression with monotonic function and query contains comparison with constant whose type is different. This fixes . (). Fix potentially low performance and slightly incorrect result for `uniqExact`, `topK`, `sumDistinct` and similar aggregate functions called on Float types with NaN values. It also triggered assert in debug build. This fixes . (). The server may crash if user passed specifically crafted arguments to the function `h3ToChildren`. This fixes . (). Fix possible error `Totals having transform was already added to pipeline` in case of a query from delayed replica. (). Fix crash in `LEFT ASOF JOIN` with `joinusenulls=1`. (). Throw error on `arrayJoin()` function in `JOIN ON` section. (). Fix segfault when function `groupArrayMovingSum` deserializes empty state. Fixes . (). Fixed the deadlock in textlog. It is a part of . This fixes . (). Fix PrettyCompactMonoBlock for clickhouse-local. Fix extremes/totals with PrettyCompactMonoBlock. Fixes . (). Fix queries with constant columns and `ORDER BY` prefix of primary key.

Fix empty output for `Arrow` and `Parquet` formats in case if query return zero rows. It was done because empty output is not valid for this formats. (). Fix `aggregate function any(x) is found inside another aggregate function in query` error with `SET optimizemovefunctionsoutof_any = 1` and aliases inside `any()`. (). Fix invalid return type for comparison of tuples with `NULL` elements. Fixes . (). Fix error in `parseDateTimeBestEffort` function when unix timestamp was passed as an argument. This fixes . (). Add compiler option to control that stack frames are not too large. This will help to run the code in fibers with small stack size. (). Rework integrational configuration paths. (). Add docker image for fast tests. (). Update fmtlib to master (7.0.1). (). Add PEERDIR(protoc) as protobuf format parses .proto file in runtime. (). Now we check that server is able to start after stress tests run. This fixes . (). Improve performance of TestKeeper. This will speedup tests with heavy usage of Replicated tables. (). Adding RBAC syntax tests in TestFlows. (). Apply random query mutations (fuzzing) in stress tests. (). Connector-ODBC updated to mysql-connector-odbc-8.0.21. (). Fix build of test under Mac OS X. This closes . (). increasing timeouts in testflows tests. (). Add a test for `ALTER TABLE CLEAR COLUMN` query for primary key columns. (). Do not build helpercontainer image inside integrational tests. Build docker container in CI and use pre-built helpercontainer in integration tests. (). Check an ability that we able to restore the backup from an old version to the new version. This closes . (). Fix MSan error in "rdkafka" library. This closes . Updated `rdkafka` to version 1.5 (master). (). Rerun some tests in fast test check. (). Adding extra xfails for some ldap tests. (). Added tests for RBAC functionality of `SELECT` privilege in TestFlows. (). Rewrote Function tests to gtest. Removed useless includes from tests. (). Improve cache purge in documentation deploy script. (). Fixing 00960liveviewwatchevents_live.py test. (). Small fixes to the RBAC SRS. (). Even more retries in zkutil gtest to prevent test flakiness. (). Add script which set labels for pull requests in GitHub hook. (). Use `shellcheck` for sh tests linting. (). Use `shellcheck` for sh tests linting. (). Fix libunwind build in AArch64. This fixes . (). Function `materialize` (the function for ClickHouse testing) will work for NULL as expected - by transforming it to non-constant column. (). Now fast test will wait server with retries. (). Applying LDAP authentication test fixes. (). Fix timeout error during server restart in the stress test. (). - Added testing for RBAC functionality of INSERT privilege in TestFlows. - Expanded tables on which SELECT is being tested. - Added Requirements to match new table engine tests. (). Remove some of recursive submodules. See . (). Ensure that all the submodules are from proper URLs. Continuation of . This fixes . (). Avoid re-loading completion from the history file after each query (to avoid history overlaps with other client sessions). (). NO CL ENTRY: 'Bump numpy from 1.18.5 to 1.19.1 in /docs/tools'. (). NO CL ENTRY: 'Bump urllib3 from 1.25.9 to 1.25.10 in /docs/tools'. (). NO CL ENTRY: 'Revert "Refactor joinGet and implement multi-key lookup."'. (). NO CL ENTRY: 'Revert "Abort on std::outofrange in debug builds"'. (). NO CL ENTRY: 'Bump protobuf from 3.12.2 to 3.12.4 in /docs/tools'. (). NO CL ENTRY: 'Merge '. (). NO CL ENTRY: 'Revert "Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQuer"'. ().