Project: StarRocks (App Definition and Development / Database)
File: release-2.2.md

displayed_sidebar: "English" Release date: April 6, 2023 Optimized the bitmap_contains() function to reduce its memory consumption and improve its performance in some scenarios. Optimized the Compaction framework to reduce its CPU resource consumption. The following bugs are fixed: If the requested URL in a Stream Load job is not correct, the responsible FE hangs and is unable to handle the HTTP request. When the responsible FE collects statistics, it may consume an abnormally large amount of memory, which causes OOM. BEs crash if memory release is not properly handled in some queries. After the command TRUNCATE TABLE is executed, a NullPointerException may occur and the responsible FE fails to restart. Release date: December 2, 2022 Optimized the error message returned for Routine Load jobs. Supports the logical operator `&&`. Queries are immediately canceled when the BE crashes, preventing system stuck issues caused by expired queries. Optimized the FE start script. Java version is now checked during FE start. Supports deleting large volumes of data from Primary Key tables. The following bugs are fixed: When users create a view from multiple tables (UNION), BEs crash if the leftmost child of UNION operations uses NULL constants. () BEs crash if the Parquet file to query has inconsistent column types with Hive table schema. When a query contains a large number of OR operators, the planner needs to perform excessive recursive calculations, which causes the query to time out. The query result is incorrect when the subquery contains a LIMIT clause. The CREATE VIEW statement fails when double quotation marks in the SELECT clause are mixed with single quotation marks. Release date: November 15, 2022 Added the session variable `hivepartitionstatssamplesize` to control the number of Hive partitions from which to collect statistics. An excessive number of partitions will cause errors in obtaining Hive metadata. Elasticsearch external tables support custom time zones. The following bugs are fixed: The DECOMMISSION operation is stuck if an error occurs during metadata synchronization for external tables. Compaction crashes if a column that is newly added is deleted. SHOW CREATE VIEW does not display the comments that were added when creating the view. Memory leak in Java UDF may cause OOM. The node alive status stored in Follower FEs is not accurate in some scenarios because the status depends on `heartbeatRetryTimes`. To fix this issue, a property `aliveStatus` is added to `HeartbeatResponse` to indicate the node alive status. Extended the length of Hive STRING columns that can be queried by StarRocks from 64 KB to 1 MB. If a STRING column exceeds 1 MB, it will be processed as a null column during

Release date: October 17, 2022 The following bugs are fixed: BEs may crash if an expression encounters an error in the initialization stage. BEs may crash if invalid JSON data is loaded. Parallel writing encounters an error when the pipeline engine is enabled. BEs crash when the ORDER BY NULL LIMIT clause is used. BEs crash if the Parquet file to query has inconsistent column type with Hive table schema. Release date: September 23, 2022 The following bugs are fixed: Data may be lost when users load JSON data into StarRocks. The output from SHOW FULL TABLES is incorrect. In previous versions, to access data in a view, users must have permissions on both the base tables and the view. In the current version, users are only required to have permissions on the view. The result from a complex query that is nested with EXISTS or IN is incorrect. REFRESH EXTERNAL TABLE fails if the schema of the corresponding Hive table is changed. An error may occur when a non-leader FE replays the bitmap index creation operation. [#11261]( Release date: September 14, 2022 The following bugs are fixed: The result of `order by... limit...offset` is incorrect when the subquery contains LIMIT. The BE crashes if partial update is performed on a table with large data volume. Compaction causes BEs to crash if the size of BITMAP data to compact exceeds 2 GB. The like() and regexp() functions do not work if the pattern length exceeds 16 KB. The format used to represent JSON values in an array in the output is modified. Escape characters are no longer used in the returned JSON values. Release date: August 18, 2022 Improved the system performance when the pipeline engine is enabled. Improved the accuracy of memory statistics for index metadata. The following bugs are fixed: BEs may be stuck in querying Kafka partition offsets (`getpartitionoffset`) during Routine Load. An error occurs when multiple Broker Load threads attempt to load the same HDFS file. Release date: August 3, 2022 Supports synchronizing schema changes on Hive table to the corresponding external table. Supports loading ARRAY data in Parquet files via Broker Load. The following bugs are fixed: Broker Load cannot handle Kerberos logins with multiple keytab files. Supervisor may fail to restart services if stop_be.sh exits immediately after it is executed. Incorrect Join Reorder precedence causes error "Column cannot be resolved". Release date: July 24, 2022 The following bugs are fixed: An error occurs when users delete a resource group. Thrift server exits when the number of threads is insufficient. In some scenarios, join reorder in CBO returns no results. Release date: June 29, 2022 UDFs can be used across databases. Optimized concurrency control for internal processing such as schema change. This reduces pressure on FE metadata

In addition, the possibility that load jobs may pile up or slow down is reduced in scenarios where huge volume of data needs to be loaded at high concurrency. The following bugs are fixed: The number of replicas (`replication_num`) created by using CTAS is incorrect. Metadata may be lost after ALTER ROUTINE LOAD is performed. Runtime filters fail to be pushed down. Pipeline issues that may cause memory leaks. Deadlock may occur when a Routine Load job is aborted. Some profile statistics information is inaccurate. The getjsonstring function incorrectly processes JSON arrays. Release date: June 2, 2022 Optimized the data loading performance and reduced long tail latency by reconstructing part of the hotspot code and reducing lock granularity. Added the CPU and memory usage information of the machines on which BEs are deployed for each query to the FE audit log. Supported JSON data types in the Primary Key tables and Unique Key tables. Reduced FEs load by reducing lock granularity and deduplicating BE report requests. Optimized the report performance when a large number of BEs are deployed, and solved the issue of Routine Load tasks getting stuck in a large cluster. The following bugs are fixed: An error occurs when StarRocks parses the escape characters specified in the `SHOW FULL TABLES FROM DatabaseName` statement. FE disk space usage rises sharply (Fix this bug by rolling back the BDBJE version). BEs become faulty because relevant fields cannot be found in the data returned after columnar scanning is enabled (`enabledocvaluescan=true`). Release date: May 22, 2022 [Preview] The resource group management feature is released. This feature allows StarRocks to isolate and efficiently use CPU and memory resources when StarRocks processes both complex queries and simple queries from different tenants in the same cluster. [Preview] A Java-based user-defined function (UDF) framework is implemented. This framework supports UDFs that are compiled in compliance with the syntax of Java to extend the capabilities of StarRocks. [Preview] The Primary Key table supports updates only to specific columns when data is loaded to the Primary Key table in real-time data update scenarios such as order updates and multi-stream joins. [Preview] JSON data types and JSON functions are supported. External tables can be used to query data from Apache Hudi. This further improves users' data lake analytics experience with StarRocks. For more information, see . The following functions are added: ARRAY functions: , arraysort, arraydistinct, arrayjoin, reverse, arrayslice, arrayconcat, arraydifference, arraysoverlap, and arrayintersect BITMAP functions: bitmapmax and bitmapmin Other functions: retention and square The parser and analyzer of the cost-based optimizer (CBO) are restructured, the code structure is optimized, and syntaxes such as INSERT with Common Table Expression (CTE) are

These improvements are made to increase the performance of complex queries, such as queries that involve the reuse of CTEs. The performance of queries on Apache Hive external tables that are stored in cloud object storage services such as AWS Simple Storage Service (S3), Alibaba Cloud Object Storage Service (OSS), and Tencent Cloud Object Storage (COS) is optimized. After the optimization, the performance of object storage-based queries is comparable to that of HDFS-based queries. Additionally, late materialization of ORC files is supported, and queries on small files are accelerated. For more information, see . When queries from Apache Hive are run by using external tables, StarRocks automatically performs incremental updates to the cached metadata by consuming Hive metastore events such as data changes and partition changes. StarRocks also supports queries on data of the DECIMAL and ARRAY types from Apache Hive. For more information, see . The UNION ALL operator is optimized to run 2 to 25 times faster than before. A pipeline engine that supports adaptive parallelism and provides optimized profiles is released to improve the performance of simple queries in high concurrency scenarios. Multiple characters can be combined and used as a single row delimiter for CSV files that are to be imported. Deadlocks occur if data is loaded or changes are committed into tables that are based on the primary Key table. Frontends (FEs), including FEs that run Oracle Berkeley DB Java Edition (BDB JE), are unstable. , , The result that is returned by the SUM function encounters an arithmetic overflow if the function is invoked on a large amount of data. The precision of the results that are returned by the ROUND and TRUNCATE functions is unsatisfactory. A few bugs are detected by Synthesized Query Lancer (SQLancer). For more information, see . Flink-connector-starrocks supports Apache Flink v1.14. If you use a StarRocks version later than 2.0.4 or a StarRocks version 2.1.x later than 2.1.6, you can disable the tablet clone feature before the upgrade (`ADMIN SET FRONTEND CONFIG ("maxschedulingtablets" = "0");` and `ADMIN SET FRONTEND CONFIG ("maxbalancingtablets" = "0");`). After the upgrade, you can enable this feature (`ADMIN SET FRONTEND CONFIG ("maxschedulingtablets" = "2000");` and `ADMIN SET FRONTEND CONFIG ("maxbalancingtablets" = "100");`). To roll back to the previous version that was used before the upgrade, add the `ignoreunknownlogid` parameter to the **fe.conf** file of each FE and set the parameter to `true`. The parameter is required because new types of logs are added in StarRocks v2.2.0. If you do not add the parameter, you cannot roll back to the previous version. We recommend that you set the `ignoreunknownlogid` parameter to `false` in the fe.conf file of each FE after checkpoints are created. Then, restart the FEs to restore the FEs to the previous configurations.