Project: StarRocks (App Definition and Development / Database)
File: data_cache.md

displayed_sidebar: "English" This topic describes the working principles of Data Cache and how to enable Data Cache to improve query performance on external data. In data lake analytics, StarRocks works as an OLAP engine to scan data files stored in external storage systems, such as HDFS and Amazon S3. The I/O overhead increases as the number of files to scan increases. In addition, in some ad hoc scenarios, frequent access to the same data doubles I/O overhead. To optimize the query performance in these scenarios, StarRocks 2.5 provides the Data Cache feature. This feature splits data in an external storage system into multiple blocks based on predefined policies and caches the data on StarRocks backends (BEs). This eliminates the need to pull data from external systems for each access request and accelerates queries and analysis on hot data. Data Cache only works when you query data from external storage systems by using external catalogs or external tables (excluding external tables for JDBC-compatible databases). It does not work when you query StarRocks native tables. StarRocks splits data in an external storage system into multiple blocks of the same size (1 MB by default), and caches the data on BEs. Block is the smallest unit of data cache, which is configurable. For example, if you set the block size to 1 MB and you want to query a Parquet file of 128 MB from Amazon S3, StarRocks splits the file into 128 blocks. The blocks are [0, 1 MB), [1 MB, 2 MB), [2 MB, 3 MB) ... [127 MB, 128 MB). StarRocks assigns a globally unique ID to each block, called a cache key. A cache key consists of the following three parts. ```Plain hash(filename) + fileModificationTime + blockId ``` The following table provides descriptions of each part. | Component item | Description | | | | | filename | The name of the data file. | | fileModificationTime | The last modification time of the data file. | | blockId | The ID that StarRocks assigns to a block when splitting the data file. The ID is unique under the same data file but is not unique within your StarRocks cluster. | If the query hits the [1 MB, 2 MB) block, StarRocks performs the following operations: Check whether the block exists in the cache. If the block exists, StarRocks reads the block from the cache. If the block does not exist, StarRocks reads the block from Amazon S3 and caches it on a BE. After Data Cache is enabled, StarRocks caches data blocks read from external storage systems. If you do not want to cache such data blocks, run the following command: ```SQL SET enablepopulatedatacache = false; ``` For more information about `enablepopulatedatacache`, see . StarRocks uses the memory and disks of BE machines to cache blocks. It supports cache solely on memory or on both the memory and disks. If you use disks as the storage media, the cache speed is directly affected by the performance of disks. Therefore, we recommend that you use high-performance disks such as NVMe disks for data cache. If you do not have high-performance disks, you can add more disks to relieve disk I/O pressure. StarRocks uses the ) (LRU) policy to cache and discard data. StarRocks first reads data from memory. If the data is not found in memory, StarRocks will read the data from disks and try to load the data read from disks into memory. Data discarded from memory will be written to disks. Data discarded from disks will be deleted. Data Cache is disabled by

To enable this feature, configure FEs and BEs in your StarRocks cluster. You can enable Data Cache for FEs by using one of the following methods: Enable Data Cache for a given session based on your requirements. ```SQL SET enablescandatacache = true; ``` Enable Data Cache for all active sessions. ```SQL SET GLOBAL enablescandatacache = true; ``` Add the following parameters to the conf/be.conf file of each BE. Then restart each BE to make the settings take effect. | Parameter | Description | Default value | | - | | -| | datacache_enable | Whether to enable Data Cache.<ul><li>`true`: Data Cache is enabled.</li><li>`false`: Data Cache is disabled.</li></ul> | false | | datacachediskpath | The paths of disks. You can configure more than one disk and separate the disk paths with semicolons (;). We recommend that the number of paths you configured be the same as the number of disks of your BE machine. When the BE starts, StarRocks automatically creates a disk cache directory (the creation fails if no parent directory exists). | `${STARROCKS_HOME}/datacache` | | datacachemetapath | The storage path of block metadata. You can leave this parameter unspecified. | `${STARROCKS_HOME}/datacache` | | datacachememsize | The maximum amount of data that can be cached in the memory. You can set it as a percentage (for example, `10%`) or a physical limit (for example, `10G`, `21474836480`). We recommend that you set the value of this parameter to at least 10 GB. | `10%` | | datacachedisksize | The maximum amount of data that can be cached in a single disk. You can set it as a percentage (for example, `80%`) or a physical limit (for example, `2T`, `500G`). For example, if you configure two disk paths for the `datacachediskpath` parameter and set the value of the `datacachedisksize` parameter to `21474836480` (20 GB), a maximum of 40 GB data can be cached in these two disks. | `0`, which indicates that only the memory is used to cache data. | Examples of setting these parameters. ```Plain datacache_enable = true datacachediskpath = /home/disk1/sr/dlacachedata/;/home/disk2/sr/dlacachedata/ datacachememsize = 2147483648 datacachedisksize = 1288490188800 ``` You can check whether a query hits data cache by analyzing the following metrics in the query profile: `DataCacheReadBytes`: the amount of data that StarRocks reads directly from its memory and disks. `DataCacheWriteBytes`: the amount of data loaded from an external storage system to StarRocks' memory and disks. `BytesRead`: the total amount of data that is read, including data that StarRocks reads from an external storage system, its memory, and disks. Example 1: In this example, StarRocks reads a large amount of data (7.65 GB) from the external storage system and only a few data (518.73 MB) from the memory and disks. This means that few data caches were hit. ```Plain Table: lineorder DataCacheReadBytes: 518.73 MB MAXOFDataCacheReadBytes: 4.73 MB MINOFDataCacheReadBytes: 16.00 KB DataCacheReadCounter: 684 MAXOFDataCacheReadCounter: 4 MINOFDataCacheReadCounter: 0 DataCacheReadTimer: 737.357us DataCacheWriteBytes: 7.65 GB MAXOFDataCacheWriteBytes: 64.39 MB MINOFDataCacheWriteBytes: 0.00 DataCacheWriteCounter: 7.887K (7887) MAXOFDataCacheWriteCounter: 65 MINOFDataCacheWriteCounter: 0 DataCacheWriteTimer: 23.467ms MAXOFDataCacheWriteTimer: 62.280ms MINOFDataCacheWriteTimer: 0ns BufferUnplugCount: 15 MAXOFBufferUnplugCount: 2 MINOFBufferUnplugCount: 0 BytesRead: 7.65 GB MAXOFBytesRead: 64.39 MB MINOFBytesRead: 0.00 ``` Example 2: In this example, StarRocks reads a large amount of data (46.08 GB) from data cache and no data from the external storage system, which means StarRocks reads data only from data cache. ```Plain Table: lineitem DataCacheReadBytes: 46.08 GB MAXOFDataCacheReadBytes: 194.99 MB MINOFDataCacheReadBytes: 81.25 MB DataCacheReadCounter: 72.237K (72237) MAXOFDataCacheReadCounter: 299 MINOFDataCacheReadCounter: 118 DataCacheReadTimer: 856.481ms MAXOFDataCacheReadTimer: 1s547ms MINOFDataCacheReadTimer: 261.824ms DataCacheWriteBytes: 0.00 DataCacheWriteCounter: 0 DataCacheWriteTimer: 0ns BufferUnplugCount: 1.231K (1231) MAXOFBufferUnplugCount: 81 MINOFBufferUnplugCount: 35 BytesRead: 46.08 GB MAXOFBytesRead: 194.99 MB MINOFBytesRead: 81.25 MB ```