Project: StarRocks (App Definition and Development / Database)
File: release-2.4.md

displayed_sidebar: "English" Release date: April 21, 2023 Forbade the List Partition syntax because it may cause errors in upgrading metadata. Supports BITMAP, HLL, and PERCENTILE types for materialized views. Optimized the inference of `storagemedium`. When BEs use both SSD and HDD as storage devices, if the property `storagecooldowntime` is specified, StarRocks sets `storagemedium` to `SSD`. Otherwise, StarRocks sets `storage_medium` to `HDD`. Optimized the accuracy of thread dump. Optimized load efficiency by triggering metadata compaction before loading. Optimized the Stream Load planner timeout. Optimized the Unique Key table performance by forbidding the collection of statistics from value columns. The following bugs are fixed: NPE is returned when an unsupported data type is used in CREATE TABLE. Wrong results are returned to queries using Broadcast Join with the short-circuit. The disk occupation problem caused by the wrong data truncate logic. The AuditLoader plugin can neither be installed nor deleted. If an exception is thrown when a tablet is being scheduled, other tablets in the same batch will never be scheduled. Unknown Error is returned when an unsupported SQL function is used in the creation of a synchronous materialized view. Multiple COUNT DISTINCT calculations are incorrectly rewritten. Wrong results are returned to queries on the tablet that is in compaction. Wrong results are returned to queries with aggregation. No error message is returned when loading NULL parquet data into NOT NULL columns. The query concurrency metric decreases slowly when the concurrency limit of a resource group is continuously reached. FE fails to start when replaying the `InsertOverwriteJob` state change log. The Primary Key table deadlock. For Colocation tables, the replica status can be manually specified as `bad` by using statements like `ADMIN SET REPLICA STATUS PROPERTIES ("tabletid" = "10003", "backendid" = "10001", "status" = "bad");`. If the number of BEs is less than or equal to the number of replicas, the corrupted replica cannot be repaired. Issues caused by ARRAY-related functions. Release date: February 22, 2023 Supports fast cancellation of load. Optimized the CPU usage of the compaction framework. Supports cumulative compaction on tablets with missing versions. The following bugs are fixed: Excessive dynamic partitions are created when an invalid DATE value is specified. Failure to connect to Elasticsearch external tables with the default HTTPS port. BE failed to cancel a Stream Load transaction after the transaction times out. Wrong query results are returned from a local shuffle aggregation on a single BE. Queries may fail with the error message "waitforversion version: failed: apply stopped". Wrong query results are returned because OLAP scan bucket expressions are not correctly cleared. The bucket number of dynamic partitioned tables in a Colocate Group cannot be modified and an error message is returned. When you connect to a non-Leader FE node and send the SQL statement `USE

the non-Leader FE node forwards the SQL statement, with `<catalogname>` excluded, to the Leader FE node. As a result, the Leader FE node chooses to use the `defaultcatalog` and eventually fails to find the specified database. Incorrect logic for the dictmapping check before rewriting. If an FE sends an occasional heartbeat to a BE, and the heartbeat connection times out, the FE considers the BE unavailable, leading to transaction failures on the BE. `getappliedrowsets` failed for queries on newly cloned tablets on a follower FE. NPE caused by executing `SET variable = default` on a follower FE. Expressions in projection are not rewritten by the dictionary. Incorrect logic for creating dynamic partitions on weekly basis. Wrong query results are returned from a local shuffle. Incremental clones may fail. In some cases, CBO may use incorrect logic to compare whether two operators are equivalent. Failed to access JuiceFS because the JuiceFS schema is not checked and resolved properly. Release date: January 19, 2023 StarRocks checks whether the corresponding database and table exist during an Analyze to prevent NPE. Columns with data types that are not supported are not materialized for queries on external tables. Adds Java version check for the FE start script start_fe.sh. The following bugs are fixed: Stream Load may fail when timeout is not set. bRPC Send crashes when memory usage is high. StarRocks fails to load data in external tables from a StarRocks instance of an early version. Materialized view Refresh failure may cause memory leak. Schema Change hangs at the Publish stage. Memory leak caused by a materialized view QeProcessorImpl issue. The results of queries with `limit` are inconsistent. Memory leak caused by INSERT. Primary Key tables executes Tablet Migration. Broker Kerberos tickets timeout during Broker Load. The `nullable` information is inferred incorrectly in the view of a table. Modified the default backlog of Thrift Listen to `1024`. Added SQL mode `FORBIDINVALIDDATES`. This SQL mode is disabled by default. When it is enabled, StarRocks verifies the input of the DATE type, and returns an error when the input is invalid. Release date: December 14, 2022 Optimized the performance of Bucket Hint when a multitude of buckets exist. The following bugs are fixed: Flushing the Primary Key index may cause BE to crash. Materialized view types cannot be correctly identified by `SHOW FULL TABLES`. Upgrading StarRocks v2.2 to v2.4 may cause BE to crash. Broker Load may cause BE to crash. The session variable `statisticcollectparallel` does not take effect. INSERT INTO may cause BE to crash. JAVA UDF may cause BE to crash. Cloning replicas during partial updates may cause BE to crash and fail to restart. Colocated Join may not take effect. Constrained the session variable `query_timeout` with an upper limit of `259200` and a lower limit of

Deprecated the FE parameter `defaultstoragemedium`. The storage medium of a table is automatically inferred by the system. Release date: November 14, 2022 Supports non-equi joins - LEFT SEMI JOIN and ANTI JOIN. Optimized the JOIN function. Supports property `aliveStatus` in `HeartbeatResponse`. `aliveStatus` indicates if a node is alive in the cluster. Mechanisms that judge the `aliveStatus` are further optimized. Optimized the error message of Routine Load. BE crashes after being upgraded from v2.4.0RC to v2.4.0. Late materialization causes incorrect results to queries on data lakes. The getjsonint function throws exceptions. Data may be inconsistent after deletion from a PRIMARY KEY table with a persistent index. BE may crash during compaction on a PRIMARY KEY table. The json_object function returns incorrect results when its input contains an empty string. BE crashes due to `RuntimeFilter`. FE hangs due to excessive recursive computations in CBO. BE may crash or report an error when exiting gracefully. Compaction crashes after data is deleted from a table with new columns added to it. Data may be inconsistent due to incorrect mechanisms in OLAP external table metadata synchronization. When one BE crashes, the other BEs may execute relevant queries till timeout. When parsing Hive external table fails, StarRocks throws error messages instead of converting relevant columns into NULL columns. Release date: October 20, 2022 Supports creating asynchronous materialized views based on multiple base tables to accelerate queries with JOIN operations. Asynchronous materialized views support all . For more information, see . Supports overwriting data via INSERT OVERWRITE. For more information, see . . Outer Join supports non-equi joins in which join items are related by comparison operators including `<`, `<=`, `>`, `>=`, and `<>`. For more information, see . Supports creating Iceberg catalogs and Hudi catalogs, which allow direct queries on data from Apache Iceberg and Apache Hudi. For more information, see and . Supports querying ARRAY-type columns from Apache Hive tables in CSV format. For more information, see . Supports viewing the schema of external data via DESC. For more information, see . Supports granting a specific role or IMPERSONATE permission to a user via GRANT and revoking them via REVOKE, and supports executing an SQL statement with IMPERSONATE permission via EXECUTE AS. For more information, see , , and . Supports FDQN access: now you can use domain name or the combination of hostname and port as the unique identification of a BE or an FE node. This prevents access failures caused by changing IP addresses. For more information, see . flink-connector-starrocks supports Primary Key table partial update. For more information, see . Provides the following new functions: arraycontainsall: checks whether a specific array is a subset of another. For more information, see . percentile_cont: calculates the percentile value with linear interpolation. For more information, see

The Primary Key table supports flushing VARCHAR-type primary key indexes to disks. From version 2.4.0, the Primary Key table supports the same data types for primary key indexes regardless of whether persistent primary key index is turned on or not. Optimized the query performance on external tables. Supports late materialization during queries on external tables in Parquet format to optimize the query performance on data lakes with small-scale filtering involved. Small I/O operations can be merged to reduce the delay for querying data lakes, thereby improving the query performance on external tables. Optimized the performance of window functions. Optimized the performance of Cross Join by supporting predicate pushdown. Histograms are added to CBO statistics. Full statistics collection is further optimized. For more information, see . Adaptive multi-threading is enabled for tablet scanning to reduce the dependency of scanning performance on the tablet number. As a result, you can set the number of buckets more easily. For more information, see . Supports querying compressed TXT files in Apache Hive. Adjusted the mechanisms of default PageCache size calculation and memory consistency check to avoid OOM issues during multi-instance deployments. Improved the performance of large-size batch load on Primary Key tables up to two times by removing final_merge operations. Supports a Stream Load transaction interface to implement two-phase commit (2PC) for transactions that are run to load data from external systems such as Apache Flink and Apache Kafka, improving the performance of highly concurrent stream loads. Functions: You can use multiple COUNT(DISTINCT) in one statement. For more information, see . Window functions min() and max() support sliding windows. For more information, see . Optimized the performance of the window_funnel function. For more information, see . The following bugs are fixed: DECIMAL data types returned by DESC are different from those specified in the CREATE TABLE statement. FE metadata management issues that affect the stability of FEs. Data load-related issues: Broke Load fails when ARRAY columns are specified. Replicas are inconsistent after data is loaded to a non-Duplicate Key table via Broker Load. Executing ALTER ROUTINE LOAD raises NPE. Data Lake analytics-related issues: Queries on Parquet data in Hive external tables fail. Incorrect results are returned for queries with `limit` clause on Elasticsearch external table. An unknown error is raised during queries on an Apache Iceberg table with a complex data type. Metadata is inconsistent between the Leader FE and Follower FE nodes. BE crashes when the size of BITMAP data exceeds 2 GB. Page Cache is enabled by default ("disablestoragepagecache" = "false"). The default cache size (`storagepagecachelimit`) is 20% of the system memory. CBO is enabled by default. Deprecated the session variable `enable_cbo`. Vectorized engine is enabled by default. Deprecated the session variable `vectorizedengineenable`. Announcing stable release of Resource Group. Announcing stable release of JSON data type and its related functions.