Project: ClickHouse (App Definition and Development / Database)
File: v20.11.1.5109-prestable.md

Make rankCorr function return nan on insufficient data . (). Aggregate functions `boundingRatio`, `rankCorr`, `retention`, `timeSeriesGroupSum`, `timeSeriesGroupRateSum`, `windowFunnel` were erroneously made case-insensitive. Now their names are made case sensitive as designed. Only functions that are specified in SQL standard or made for compatibility with other DBMS or functions similar to those should be case-insensitive. (). Remove `ANALYZE` and `AST` queries, and make the setting `enabledebugqueries` obsolete since now it is the part of full featured `EXPLAIN` query. (). Restrict to use of non-comparable data types (like `AggregateFunction`) in keys (Sorting key, Primary key, Partition key, and so on). (). If some `profile` was specified in `distributed_ddl` config section, then this profile could overwrite settings of `default` profile on server startup. It's fixed, now settings of distributed DDL queries should not affect global server settings. (). #WelchTTest aggregate function implementation. (). New functions `encrypt`, `aesencryptmysql`, `decrypt`, `aesdecryptmysql`. These functions are working slowly (below ClickHouse standards), so we consider it as an experimental feature. (). - Added support of LDAP as a user directory for locally non-existent users. (). Added `disable_merges` option for volumes in multi-disk configuration. (). Added initial OpenTelemetry support. ClickHouse now accepts OpenTelemetry traceparent headers over Native and HTTP protocols, and passes them downstream in some cases. The trace spans for executed queries are saved into the `system.opentelemetryspanlog` table. (). Allows to read/write Single protobuf message at once (w/o length-delimiters). (). Add function `formatReadableTimeDelta` that format time delta to human readable string ... (). Add `tid` and `logTrace` function. This closes . (). Add a new option `printqueryid` to clickhouse-client. It helps generate arbitrary strings with the current query id generated by the client. (). Allow specify primary key in column list of CREATE TABLE query. (). Added setting datetimeoutput_format. (). Implement ``` OFFSET offsetrowcount {ROW | ROWS} FETCH {FIRST | NEXT} fetchrowcount {ROW | ROWS} {ONLY | WITH TIES} ``` in select Query with order by. related issue:. (). Added an aggregate function, which calculates the p-value used for Welch's t-test. (). Add maxconcurrentqueriesforall_users setting, see for use cases. (). Added minimal web UI to ClickHouse. (). Added function `untuple` which is a special function which can introduce new columns to the SELECT list by flattening a named tuple. (). Added toUUIDOrNull, toUUIDOrZero cast functions. (). Add `system.replicated_fetches` table which shows currently running background fetches. (). - `errorCodeToName()` function - return variable name of the error (useful for analyzing querylog and similar) - `system.errors` table - shows how many times errors has been happened (respects `systemeventsshowzero_values`). (). Ability to create a docker image on the top of alpine. Uses precompiled binary and glibc components from ubuntu 20.04. (). Add `logqueriesminquerydurationms`, only queries slower then the value of this setting will go to `querylog`/`querythreadlog` (i.e. something like `slowquerylog` in mysql). (). > Add farmFingerprint64 function. (). Now we can provide identifiers via query parameters. And these parameters can be used as table objects or columns. . (). Added big integers (UInt256, Int128, Int256) and UUID data types support for MergeTree BloomFilter index. (). Speedup hashed/sparse_hashed dictionary loading by preallocating the hash table. (). Do not merge parts across partitions in SELECT FINAL. (). Improved performance of merges assignment in MergeTree table engines. Shouldn't be visible for the user. (). Improve performance of logical functions a little. (). Improve performance of `quantileMerge`. In previous versions it was obnoxiously slow. This closes .

Improve performance of `-OrNull` and `-OrDefault` aggregate functions. (). Allow explicitly specify columns list in `CREATE TABLE table AS table_function(...)` query. Fixes Fixes . (). Now trivial count optimization becomes slightly non-trivial. Predicates that contain exact partition expr can be optimized too. This also fixes which returns wrong count when `maxparallelreplicas > 1`. (). Enable parsing enum values by their ids for CSV, TSV and JSON input formats. (). Add reconnects to `zookeeper-dump-tree` tool. (). Remove `MemoryTrackingInBackground` metrics to avoid potentially misleading results. This fixes . (). Change level of some log messages from information to debug, so information messages will not appear for every query. This closes . (). Fix query hang (endless loop) in case of misconfiguration (`connectionswithfailovermaxtries` set to 0). (). Added boost::programoptions to `dbgenerator` in order to increase its usability. This closes . (). Treat `INTERVAL '1 hour'` as equivalent to `INTERVAL 1 HOUR`, to be compatible with Postgres. This fixes . (). Simplify the implementation of background tasks processing for the MergeTree table engines family. There should be no visible changes for user. (). Add support of cache layout for Redis dictionaries with complex key. (). Fix rare issue when clickhouse-client may abort on exit due to loading of suggestions. This fixes . (). Now it's allowed to execute `ALTER ... ON CLUSTER` queries regardless of the `<internal_replication>` setting in cluster config. (). - Fix memoryprofilerstep/maxuntrackedmemory for queries via HTTP (test included), and adjusting this value globally in xml config will not help either, since those settings are not applied anyway, only default (4MB) value is . - Fix queryid for the most root ThreadStatus of the http query (by initializing QueryScope after reading queryid). (). Add allownondeterministicoptimizeskipunused_shards (to allow non deterministic like rand() or dictGet() in sharding key). (). databaseatomicwaitfordropanddetach_synchronously/NO DELAY/SYNC for DROP DATABASE. (). Add support for nested data types (like named tuple) as sub-types. Fixes . (). If there are no tmp folder in the system (chroot, misconfigutation etc) clickhouse-local will create temporary subfolder in the current directory. (). Now it's possible to specify `PRIMARY KEY` without `ORDER BY` for MergeTree table engines family. Closes . (). try use cmake version for croaring instead of amalgamation.sh. (). Add totalrows/totalbytes (from system.tables) support for Set/Join table engines. (). Better diagnostics when client has dropped connection. In previous versions, `Attempt to read after EOF` and `Broken pipe` exceptions were logged in server. In new version, it's information message `Client has dropped the connection, cancel the query.`. (). Add `TablesToDropQueueSize` metric. It's equal to number of dropped tables, that are waiting for background data removal. (). Fix debug assertion in quantileDeterministic function. In previous version it may also transfer up to two times more data over the network. Although no bug existed. This fixes . (). Better read task scheduling for JBOD architecture and `MergeTree` storage. New setting `readbackoffmin_concurrency` which serves as the lower limit to the number of reading threads. (). Fixed bug for . Also minimized event loop lifetime. Added more efficient queues setup. (). Allow to fetch parts that are already committed or outdated in the current instance into the detached directory. It's useful when migrating tables from another cluster and having N to 1 shards mapping. It's also consistent with the current fetchPartition implementation. (). Add currentdatabase into querythread_log. (). Subqueries in WITH section (CTE) can reference previous subqueries in WITH section by their name.

- Improve scheduling of background task which removes data of dropped tables in `Atomic` databases. - `Atomic` databases do not create broken symlink to table data directory if table actually has no data directory. (). Now paratmeterized functions can be used in APPLY column transformer. (). Now `eventtimemicroseconds` field stores in Decimal64, not UInt64. Removed an incorrect check from Field::get(). (). Apply SETTINGS clause as early as possible. It allows to modify more settings in the query. This closes . (). Better update of ZooKeeper configuration in runtime. (). Make the behaviour of `minMap` and `maxMap` more desireable. It will not skip zero values in the result. Fixes . (). Better diagnostics on parse errors in input data. Provide row number on `Cannot read all data` errors. (). Update jemalloc to fix percpu_arena with affinity mask. (). Decrement the `ReadonlyReplica` metric when detaching read-only tables. This fixes . (). Fixed bug with globs in S3 table function, region from URL was not applied to S3 client configuration. (). Fix error `Cannot add simple transform to empty Pipe` which happened while reading from `Buffer` table which has different structure than destination table. It was possible if destination table returned empty result for query. Fixes . (). Possibility to move part to another disk/volume if the first attempt was failed. (). Fix drop of materialized view with inner table in Atomic database (hangs all subsequent DROP TABLE due to hang of the worker thread, due to recursive DROP TABLE for inner table of MV). (). Fix `select count()` inaccuracy for MaterializeMySQL. (). Fix exception `Block structure mismatch` in `SELECT ... ORDER BY DESC` queries which were executed after `ALTER MODIFY COLUMN` query. Fixes . (). Now exception will be thrown when `ALTER MODIFY COLUMN ... DEFAULT ...` has incompatible default with column type. Fixes . (). Fix possible deadlocks in RBAC. (). fixes fix difference expressions with same alias when analyze queries again. (). Fix incorrect empty result for query from `Distributed` table if query has `WHERE`, `PREWHERE` and `GLOBAL IN`. Fixes . (). Fixed `DROP TABLE IF EXISTS` failure with `Table ... doesn't exist` error when table is concurrently renamed (for Atomic database engine). Fixed rare deadlock when concurrently executing some DDL queries with multiple tables (like `DROP DATABASE` and `RENAME TABLE`) Fixed `DROP/DETACH DATABASE` failure with `Table ... doesn't exist` when concurrently executing `DROP/DETACH TABLE`. (). Fix a crash when database creation fails. (). Fix ambiguity in parsing of settings profiles: `CREATE USER ... SETTINGS profile readonly` is now considered as using a profile named `readonly`, not a setting named `profile` with the readonly constraint. This fixes . (). Fix rare segfaults when inserting into or selecting from MaterializedView and concurrently dropping target table (for Atomic database engine). (). Prevent replica hang for 5-10 mins when replication error happens after a period of inactivity. (). Allow to use direct layout for dictionaries with complex keys. (). Fix collate name & charset name parser and support `length = 0` for string type. (). Fix `ALTER MODIFY ... ORDER BY` query hang for `ReplicatedVersionedCollapsingMergeTree`. This fixes . (). Fix bug with MySQL database. When MySQL server used as database engine is down some queries raise Exception, because they try to get tables from disabled server, while it's unnecessary. For example, query `SELECT ... FROM system.parts` should work only with MergeTree tables and don't touch MySQL database at all. (). Fixes regression, e.g. indexOf(

Fix segfault in some cases of wrong aggregation in lambdas. (). Fix the `clickhouse-local` crash when trying to do `OPTIMIZE` command. Fixes . (). Fix dictGet in sharding_key (and similar places, i.e. when the function context is stored permanently). (). Fix the case when memory can be overallocated regardless to the limit. This closes . (). Fix a possible memory leak during `GROUP BY` with string keys, caused by an error in `TwoLevelStringHashTable` implementation. (). Fixed the inconsistent behaviour when a part of return data could be dropped because the set for its filtration wasn't created. (). Fix processing of very large entries in replication queue. Very large entries may appear in ALTER queries if table structure is extremely large (near 1 MB). This fixes . (). Fix async Distributed INSERT w/ preferlocalhostreplica=0 and internal_replication. (). Fix group by with totals/rollup/cube modifers and min/max functions over group by keys. Fixes . (). Fix DROP TABLE for Distributed (racy with INSERT). (). Fix double free in case of exception in function `dictGet`. It could have happened if dictionary was loaded with error. (). Specifically crafter argument of `round` function with `Decimal` was leading to integer division by zero. This fixes . (). Fix rapid growth of metadata when using MySQL Master -> MySQL Slave -> ClickHouse MaterializeMySQL Engine, and `slaveparallelworker` enabled on MySQL Slave, by properly shrinking GTID sets. This fixes . (). Now when parsing AVRO from input the LowCardinality is removed from type. Fixes . (). Fix querythreadlog.querydurationms unit. (). Calculation of `DEFAULT` expressions was involving possible name collisions (that was very unlikely to encounter). This fixes . (). The setting `maxparallelreplicas` worked incorrectly if the queried table has no sampling. This fixes . (). Simplify Sys/V init script. (). Fix illegal code style `&vector (). Check for `#pragma once` in headers. (). Check for executable bit on non-executable files. People often accidentially commit executable files from Windows. (). - Updated RBAC syntax tests. - Updated `INSERT` and `SELECT` RBAC tests. - New RBAC tests for `ALTER` privileges by @ritaank - New RBAC tests for `MATERIALIZED`, `LIVE`, and standard `VIEWS`. - New RBAC tests for public system tables. - New RBAC tests for `SHOW TABLES`. (). Refuse to build with AppleClang because it's difficult to find out version correspondence. This closes . (). Add scipy to fasttest docker. (). Add MySQL BinLog file check tool for MaterializeMySQL database engine. (). Add flaky check for stateless tests. (). Converting test tests/queries/0stateless/01446jsonstringseach_row to a shell script. (). - None. (). Fixing fails in LDAP external user directory tests. (). During config removal wrong exit code was expected. (). Fix LDAP tests by grabbing log size after container is stopped. (). Improve generation of build files for `ya.make` build system (Arcadia). (). Use only |name_parts| as primary name source and auto-generate full name. (). Rename struct NullSink from ReadHelpers to NullOutput, because class NullSink exists in Processors/NullSink.h. It's needed to prevent redefinition of 'NullSink' error. (). Try to make MergeTreeWriteAheadLog forward compatible. (). NO CL ENTRY: 'Revert "Write structure of table functions to metadata"'. (). NO CL ENTRY: 'Revert "scipy"'. (). NO CL ENTRY: 'Bump markdown from 3.2.1 to 3.3.2 in /docs/tools'. (). NO CL ENTRY: 'Revert "Optionally upload clickhouse binary in fast test"'. (). NO CL ENTRY: 'minor fix.'. (). NO CL ENTRY: 'Bump tornado from 5.1.1 to 6.1 in /docs/tools'. (). NO CL ENTRY: 'Bump mkdocs-macros-plugin from 0.4.17 to 0.4.20 in /docs/tools'. ().