Project: ClickHouse (App Definition and Development / Database)
File: v20.8.1.4513-prestable.md

Remove support for the `ODBCDriver` input/output format. This was a deprecated format once used for communication with the ClickHouse ODBC driver, now long superseded by the `ODBCDriver2` format. Resolves . (). Extend `paralleldistributedinsert_select` setting, adding an option to run `INSERT` into local table. The setting changes type from `Bool` to `UInt64`, so the values `false` and `true` are no longer supported. If you have these values in server configuration, the server will not start. Please replace them with `0` and `1`, respectively. (). Now `OPTIMIZE FINAL` query doesn't recalculate TTL for parts that were added before TTL was created. Use `ALTER TABLE ... MATERIALIZE TTL` once to calculate them, after that `OPTIMIZE FINAL` will evaluate TTL's properly. This behavior never worked for replicated tables. (). Support `MaterializeMySQL` database engine. Implements . (). Support Kerberos authentication in Kafka, using `krb5` and `cyrus-sasl` libraries. (). Add types `Int128`, `Int256`, `UInt256` and related functions for them. Extend Decimals with Decimal256 (precision up to 76 digits). New types are under the setting `allowexperimentalbigint_types`. (). Function `position` now supports optional `start_pos` argument. (). Add `ALTER SAMPLE BY` statement that allows to change table sample clause. (). Add new optional section <user_directories> to the main config. (). Add function `normalizeQuery` that replaces literals, sequences of literals and complex aliases with placeholders. Add function `normalizedQueryHash` that returns identical 64bit hash values for similar queries. It helps to analyze query log. This closes . (). Add `quantileExactLow` and `quantileExactHigh` implementations with respective aliases for `medianExactLow` and `medianExactHigh`. (). Add function `defaultValueOfTypeName` that returns the default value for a given type. (). Added `date_trunc` function that truncates a date/time value to a specified date/time part. (). Add the ability to specify `Default` compression codec for columns that correspond to settings specified in `config.xml`. Implements: . (). Add setting `minindexgranularitybytes` that protects against accidentally creating a table with very low `indexgranularity_bytes` setting. (). Add `countDigits(x)` function that count number of decimal digits in integer or decimal column. Add `isDecimalOverflow(d, (). When performing trivial `INSERT SELECT` queries, automatically set `maxthreads` to 1 or `maxinsertthreads`, and set `maxblocksize` to `mininsertblocksize_rows`. Related to . (). Optimize `has()`, `indexOf()` and `countEqual()` functions for `Array(LowCardinality(T))` and constant right arguments. (). Slightly improve performance of aggregation by UInt8/UInt16 keys. (). Fail fast if `maxrowstoread` limit is exceeded on parts scan. The motivation behind this change is to skip ranges scan for all selected parts if it is clear that `maxrowstoread` is already exceeded. The change is quite noticeable for queries over big number of parts. (). Enable parallel INSERTs for table engines `Null`, `Memory`, `Distributed` and `Buffer`. (). Slightly optimize very short queries with LowCardinality. (). Added cache layer for DiskS3 (cache to local disk mark and index files). (). Fix assert when decimal has too large negative exponent. Fixes . (). Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events. (). Proper remote host checking in S3 redirects (security-related thing). (). Enable mixed granularity parts by default. (). Move parts from DIskLocal to DiskS3 in parallel. (). Support compound identifiers for custom settings. (). Provide monotonicity for `toDate/toDateTime` functions in more cases. Now the input arguments are saturated more naturally and provides better monotonicity. (). In previous versions `lcm` function may produce assertion violation in debug build if called with specifically crafted arguments. This fixes . (). Add MergeTree Write-Ahead-Log(WAL) dump tool. (). Avoid too slow queries when arrays are manipulated as fields. Throw exception instead.

Add , it works even when replication topology changes, and supported/prefered in MySQL 5.6/5.7/8.0 2. Add BIT/SET filed type supports 3. Fix up varchar type meta length bug. (). Fix data race in `lgamma` function. This race was caught only in `tsan`, no side effects a really happened. (). Function `arrayCompact` will compare NaNs bitwise if the type of array elements is Float32/Float64. In previous versions NaNs were always not equal if the type of array elements is Float32/Float64 and were always equal if the type is more complex, like Nullable(Float64). This closes . (). Better error message for null value of TabSeparatedRow format. (). Fix wrong error for long queries. It was possible to get syntax error other than `Max query size exceeded` for correct query. (). Corrected an error in AvroConfluent format that caused the Kafka table engine to stop processing messages when an abnormally small, malformed, message was received. (). Increase limit in -Resample combinator to 1M. (). Conditional aggregate functions (for example: `avgIf`, `sumIf`, `maxIf`) should return `NULL` when miss rows and use nullable arguments. (). Slightly better performance of Memory table if it was constructed from a huge number of very small blocks (that's unlikely). Author of the idea: . Closes . (). Fix `toDeicmal256()` and attempt to fix potential perf regression in Decimal to Float conversion. (). Now it's possible to `ALTER TABLE tablename FETCH PARTITION partitionexpr FROM 'zk://<host>:<port>/path-in-zookeeper'`. It's useful for shipping data to new clusters. (). subquery hash values are not enough to distinguish. . (). Removed wrong auth access check when using ClickHouseDictionarySource to query remote tables. (). Fix access to redis dictionary after connection was dropped once. It may happen with `cache` and `direct` dictionary layouts. (). Fix parsing row policies from users.xml when names of databases or tables contain dots. This fixes , . (). Fix missing or excessive headers in `TSV/CSVWithNames` formats. This fixes . (). Fix possible race in `StorageMemory`. https://clickhouse-test-reports.s3.yandex.net/0/9cac8a7244063d2092ad25d45502611e18d3749c/stresstest(thread)/stderr.log Have no idea how to write a test. (). Fix wrong code in function `netloc`. This fixes . (). Fix premature `ON CLUSTER` timeouts for queries that must be executed on a single replica. Fixes , Fixes , Fixes , Fixes . (). Fix secondary indices corruption in compact parts. (). Fixed the behaviour when sometimes cache-dictionary returned default value instead of present value from source. (). Concurrent `ALTER ... REPLACE/MOVE PARTITION ...` queries might cause deadlock. It's fixed. (). Fix typo in error message about `The value of 'numberoffreeentriesinpooltolowermaxsizeof_merge' setting`. (). Fix crash in JOIN with StorageMerge and `set enableoptimizepredicate_expression=1`. (). Do not optimize any(arrayJoin()) -> arrayJoin() under optimizemovefunctionsoutof_any. (). Fix visible data clobbering by progress bar in client in interactive mode. This fixes and and and fixes . (). Fix incorrect message in `clickhouse-server.init` while checking user and group. (). Fixes /replicas_status endpoint response status code when verbose=1. (). Fix logging Settings.Names/Values when logqueriesmintype > QUERYSTART. (). Fix race condition between DETACH and background merges. Parts may revive after detach. This is continuation of that did not fix the issue but introduced a test that started to fail in very rare cases, demonstrating the issue. (). Add range check for h3KRing function. This fixes . (). Fixed `Directory not empty` error when concurrently executing `DROP DATABASE` and `CREATE TABLE`. (). Fix step overflow in range(). (). Fix reading from MergeTree table with INDEX of type SET fails when comparing against NULL. This fixes . (). Fix topK/topKWeighted merge (with non-default parameters). (). Fix incorrect sorting for `FixedString` columns. Fixes .

Fixed potential deadlock when renaming `Distributed` table. (). Fix wrong results in select queries with `DISTINCT` keyword in case `optimizeduplicateorderbyand_distinct` setting is enabled. (). Fix parser to reject create table as table function with engine. (). Fixed wrong mount point in extra info for `Poco::Exception: no space left on device`. (). Fix pointInPolygon with const 2d array as polygon. (). Fix DistributedFilesToInsert metric (zeroed when it should not). (). When waiting for a dictionary update to complete, use the timeout specified by `querywaittimeout_milliseconds` setting instead of a hard-coded value. (). Fix formatting of minimal negative decimal numbers. This fixes . (). Fix creation of tables with named tuples. This fixes . (). Fix crash when INERT INTO Kafka engine table from an empty result set with a subquery. ... (). Fixed incorrect sorting order if LowCardinality column. This fixes . (). Fix crash in mark inclusion search introduced in https://github.com/ClickHouse/ClickHouse/pull/12277 . (). fixes fix wrong lexer in MaterializeMySQL database engine dump stage. (). Fix handling of empty transactions in `MaterializeMySQL` database engine. This fixes . (). Fix segfault in `clickhouse-odbc-bridge` during schema fetch from some external sources. This PR fixes . (). Move Dockerfiles from integration tests to `docker/test` directory. docker_compose files are available in `runner` docker container. Docker images are built in CI and not in integration tests. (). Skip PR's from robot-clickhouse. (). Fix typos in code with codespell. (). Enable check for typos in code with `codespell`. (). Ensure that there is no copy-pasted GPL code. (). Allow to run `clickhouse` binary without configuration. (). Added `clickhouse install` script, that is useful if you only have a single binary. (). Fix the remaining shellcheck notices. A preparation to enable Shellcheck. (). Enable Shellcheck in CI as a linter of .sh tests. This closes . (). Make sure is invalid. (). Increasing health-check timeouts for ClickHouse nodes and adding support to dump docker-compose logs if unhealthy containers found. (). Build ClickHouse with the most fresh tzdata from package repository. (). Removed `-DENABLECURLCLIENT` for `contrib/aws`. (). Expose version of embedded tzdata via TZDATAVERSION in system.buildoptions. (). Updating LDAP user authentication suite to check that it works with RBAC. (). Add a CMake option to fail configuration instead of auto-reconfiguration, enabled by default. (). Fix link error in shared build. (). FIx cassandra build on Mac OS. (). Added docker image for style check. Added style check that all docker and docker compose files are located in docker directory. (). ZooKeeper cannot work reliably in unit tests in CI infrastructure. Using unit tests for ZooKeeper interaction with real ZooKeeper is bad idea from the start (unit tests are not supposed to verify complex distributed systems). We already using integration tests for this purpose and they are better suited. (). Testflows LDAP module: adding missing certificates and dhparam.pem for openldap4. (). Enabled text-log in stress test to find more bugs. (). Adding retry logic when bringing up docker-compose cluster * Increasing COMPOSEHTTPTIMEOUT. (). Add the ability to write js-style comments in skip_list.json. (). Switch tests docker images to use test-base parent. (). Actually there are no symlinks there, so `-type f` is enough ``` ~/workspace/ClickHouse/contrib/cctz/testdata/zoneinfo$ find . -type l -ls | wc -l 0 ``` Closes . (). Fix readline so it dumps history to file now. (). Create `system` database with `Atomic` engine by default. (). NO CL ENTRY: 'Bump pymdown-extensions from 7.1 to 8.0 in /docs/tools'. (). NO CL ENTRY: 'Bump protobuf from 3.12.4 to 3.13.0 in /docs/tools'. ().