Project: StarRocks (App Definition and Development / Database)
File: cloud_storage_load.md

displayed_sidebar: "English" import InsertPrivNote from '../assets/commonMarkdown/insertPrivNote.md' StarRocks supports using one of the following methods to load huge amounts of data from cloud storage: and . In v3.0 and earlier, StarRocks only supports Broker Load, which runs in asynchronous loading mode. After you submit a load job, StarRocks asynchronously runs the job. You can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. Broker Load ensures the transactional atomicity of each load job that is run to load multiple data files, which means that the loading of multiple data files in one load job must all succeed or fail. It never happens that the loading of some data files succeeds while the loading of the other files fails. Additionally, Broker Load supports data transformation at data loading and supports data changes made by UPSERT and DELETE operations during data loading. For more information, see and . <InsertPrivNote /> From v3.1 onwards, StarRocks supports directly loading the data of specific file formats from AWS S3 by using the INSERT command and the FILES keyword, saving you from the trouble of creating an external table first. For more information, see . This topic focuses on using to load data from cloud storage. Broker Load supports the following data file formats: CSV Parquet ORC JSON (supported from v3.2.3 onwards) NOTE For CSV data, take note of the following points: You can use a UTF-8 string, such as a comma (,), tab, or pipe (|), whose length does not exceed 50 bytes as a text delimiter. Null values are denoted by using `\N`. For example, a data file consists of three columns, and a record from that data file holds data in the first and third columns but no data in the second column. In this situation, you need to use `\N` in the second column to denote a null value. This means the record must be compiled as `a,\N,b` instead of `a,,b`. `a,,b` denotes that the second column of the record holds an empty string. After you submit a load job to an FE, the FE generates a query plan, splits the query plan into portions based on the number of available BEs or CNs and the size of the data file you want to load, and then assigns each portion of the query plan to an available BE or CN. During the load, each involved BE or CN pulls the data of the data file from your external storage system, pre-processes the data, and then loads the data into your StarRocks cluster. After all BEs or CNs finish their portions of the query plan, the FE determines whether the load job is successful. The following figure shows the workflow of a Broker Load job. Log in to your local file system and create two CSV-formatted data files, `file1.csv` and `file2.csv`. Both files consist of three columns, which represent user ID, user name, and user score in sequence. `file1.csv` ```Plain 1,Lily,21 2,Rose,22 3,Alice,23 4,Julia,24 ``` `file2.csv` ```Plain 5,Tony,25 6,Adam,26 7,Allen,27 8,Jacky,28 ``` Upload `file1.csv` and `file2.csv` to the `input` folder of your AWS S3 bucket `buckets3`, to the `input` folder of your Google GCS bucket `bucketgcs`, to the `input` folder of your S3-compatible storage object (such as MinIO) bucket `bucket_minio`, and to the specified paths of your Azure Storage. Log in to your StarRocks database (for example, `test_db`) and create two Primary Key tables, `table1` and

Both tables consist of three columns: `id`, `name`, and `score`, of which `id` is the primary key. ```SQL CREATE TABLE `table1` ( `id` int(11) NOT NULL COMMENT "user ID", `name` varchar(65533) NULL DEFAULT "" COMMENT "user name", `score` int(11) NOT NULL DEFAULT "0" COMMENT "user score" ) ENGINE=OLAP PRIMARY KEY(`id`) DISTRIBUTED BY HASH(`id`); CREATE TABLE `table2` ( `id` int(11) NOT NULL COMMENT "user ID", `name` varchar(65533) NULL DEFAULT "" COMMENT "user name", `score` int(11) NOT NULL DEFAULT "0" COMMENT "user score" ) ENGINE=OLAP PRIMARY KEY(`id`) DISTRIBUTED BY HASH(`id`); ``` Note that Broker Load supports accessing AWS S3 according to the S3 or S3A protocol. Therefore, when you load data from AWS S3, you can include `s3://` or `s3a://` as the prefix in the S3 URI that you pass as the file path (`DATA INFILE`). Also, note that the following examples use the CSV file format and the instance profile-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other authentication methods, see . Execute the following statement to load the data of `file1.csv` stored in the `input` folder of your AWS S3 bucket `bucket_s3` into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_101 ( DATA INFILE("s3a://bucket_s3/input/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "aws.s3.useinstanceprofile" = "true", "aws.s3.region" = "<awss3region>" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of all data files (`file1.csv` and `file2.csv`) stored in the `input` folder of your AWS S3 bucket `bucket_s3` into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_102 ( DATA INFILE("s3a://bucket_s3/input/*") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "aws.s3.useinstanceprofile" = "true", "aws.s3.region" = "<awss3region>" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of `file1.csv` and `file2.csv` stored in the `input` folder of your AWS S3 bucket `bucket_s3` into `table1` and `table2`, respectively: ```SQL LOAD LABEL testdb.labelbrokerloadtest_103 ( DATA INFILE("s3a://bucket_s3/input/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) , DATA

INTO TABLE table2 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "aws.s3.useinstanceprofile" = "true", "aws.s3.region" = "<awss3region>" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1` and `table2`: Query `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Query `table2`: ```SQL SELECT * FROM table2; ++-+-+ | id | name | score | ++-+-+ | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set (0.01 sec) ``` Note that Broker Load supports accessing Google GCS only according to the gs protocol. Therefore, when you load data from Google GCS, you must include `gs://` as the prefix in the GCS URI that you pass as the file path (`DATA INFILE`). Also, note that the following examples use the CSV file format and the VM-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other authentication methods, see . Execute the following statement to load the data of `file1.csv` stored in the `input` folder of your Google GCS bucket `bucket_gcs` into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_201 ( DATA INFILE("gs://bucket_gcs/input/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "gcp.gcs.usecomputeengineserviceaccount" = "true" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of all data files (`file1.csv` and `file2.csv`) stored in the `input` folder of your Google GCS bucket `bucket_gcs` into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_202 ( DATA INFILE("gs://bucket_gcs/input/*") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "gcp.gcs.usecomputeengineserviceaccount" = "true" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this

After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of `file1.csv` and `file2.csv` stored in the `input` folder of your Google GCS bucket `bucket_gcs` into `table1` and `table2`, respectively: ```SQL LOAD LABEL testdb.labelbrokerloadtest_203 ( DATA INFILE("gs://bucket_gcs/input/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) , DATA INFILE("gs://bucket_gcs/input/file2.csv") INTO TABLE table2 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "gcp.gcs.usecomputeengineserviceaccount" = "true" ); PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1` and `table2`: Query `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Query `table2`: ```SQL SELECT * FROM table2; ++-+-+ | id | name | score | ++-+-+ | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set (0.01 sec) ``` Note that when you load data from Azure Storage, you need to determine which prefix to use based on the access protocol and specific storage service that you use: When you load data from Blob Storage, you must include `wasb://` or `wasbs://` as a prefix in the file path (`DATA INFILE`) based on the protocol that is used to access your storage account: If your Blob Storage allows access only through HTTP, use `wasb://` as the prefix, for example, `wasb://<container>@<storageaccount>.blob.core.windows.net/<path>/<filename>/*`. If your Blob Storage allows access only through HTTPS, use `wasbs://` as the prefix, for example, `wasbs://<container>@<storageaccount>.blob.core.windows``.net/<path>/<filename>/*` When you load data from Data Lake Storage Gen1, you must include `adl://` as a prefix in the file path (`DATA INFILE`), for example, `adl://<datalakestoragegen1name>.azuredatalakestore.net/<path>/<file_name>`. When you load data from Data Lake Storage Gen2, you must include `abfs://` or `abfss://` as a prefix in the file path (`DATA INFILE`) based on the protocol that is used to access your storage account: If your Data Lake Storage Gen2 allows access only via HTTP, use `abfs://` as the prefix, for example, `abfs://<container>@<storageaccount>.dfs.core.windows.net/<filename>`. If your Data Lake Storage Gen2 allows access only via HTTPS, use `abfss://` as the prefix, for example, `abfss://<container>@<storageaccount>.dfs.core.windows.net/<filename>`. Also, note that the following examples use the CSV file format, Azure Blob Storage, and the shared key-based authentication method. For information about how to load data in other formats and about the authentication parameters that you need to configure when using other Azure storage services and authentication methods, see . Execute the following statement to load the data of `file1.csv` stored in the specified path of your Azure Storage into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_301 ( DATA INFILE("wasb[s]://<container>@<storage_account>.blob.core.windows.net/<path>/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "azure.blob.storageaccount" = "<blobstorageaccountname>", "azure.blob.sharedkey" = "<blobstorageaccountshared_key>" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job

This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of all data files (`file1.csv` and `file2.csv`) stored in the specified path of your Azure Storage into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_302 ( DATA INFILE("wasb[s]://<container>@<storage_account>.blob.core.windows.net/<path>/*") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "azure.blob.storageaccount" = "<blobstorageaccountname>", "azure.blob.sharedkey" = "<blobstorageaccountshared_key>" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of `file1.csv` and `file2.csv` stored in the specified path of your Azure Storage into `table1` and `table2`, respectively: ```SQL LOAD LABEL testdb.labelbrokerloadtest_303 ( DATA INFILE("wasb[s]://<container>@<storage_account>.blob.core.windows.net/<path>/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) , DATA INFILE("wasb[s]://<container>@<storage_account>.blob.core.windows.net/<path>/file2.csv") INTO TABLE table2 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "azure.blob.storageaccount" = "<blobstorageaccountname>", "azure.blob.sharedkey" = "<blobstorageaccountshared_key>" ); PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1` and `table2`: Query `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Query `table2`: ```SQL SELECT * FROM table2; ++-+-+ | id | name | score | ++-+-+ | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set (0.01 sec) ``` The following examples use the CSV file format and the MinIO storage system. For information about how to load data in other formats, see . Execute the following statement to load the data of `file1.csv` stored in the `input` folder of your MinIO bucket `bucket_minio` into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_401 ( DATA INFILE("s3://bucket_minio/input/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "aws.s3.enable_ssl" = "false", "aws.s3.enablepathstyle_access" = "true",

= "<s3_endpoint>", "aws.s3.accesskey" = "<iamuseraccesskey>", "aws.s3.secretkey" = "<iamusersecretkey>" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of all data files (`file1.csv` and `file2.csv`) stored in the `input` folder of your MinIO bucket `bucket_minio` into `table1`: ```SQL LOAD LABEL testdb.labelbrokerloadtest_402 ( DATA INFILE("s3://bucket_minio/input/*") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) ) WITH BROKER ( "aws.s3.enable_ssl" = "false", "aws.s3.enablepathstyle_access" = "true", "aws.s3.endpoint" = "<s3_endpoint>", "aws.s3.accesskey" = "<iamuseraccesskey>", "aws.s3.secretkey" = "<iamusersecretkey>" ) PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set (0.01 sec) ``` Execute the following statement to load the data of all data files (`file1.csv` and `file2.csv`) stored in the `input` folder of your MinIO bucket `bucket_minio` into `table1` and `table2`, respectively: ```SQL LOAD LABEL testdb.labelbrokerloadtest_403 ( DATA INFILE("s3://bucket_minio/input/file1.csv") INTO TABLE table1 COLUMNS TERMINATED BY "," (id, name, score) , DATA INFILE("s3://bucket_minio/input/file2.csv") INTO TABLE table2 COLUMNS TERMINATED BY "," (id, city) ) WITH BROKER ( "aws.s3.enable_ssl" = "false", "aws.s3.enablepathstyle_access" = "true", "aws.s3.endpoint" = "<s3_endpoint>", "aws.s3.accesskey" = "<iamuseraccesskey>", "aws.s3.secretkey" = "<iamusersecretkey>" ); PROPERTIES ( "timeout" = "3600" ); ``` After you submit the load job, you can use `SELECT * FROM information_schema.loads` to query the job result. This feature is supported from v3.1 onwards. For more information, see the "" section of this topic. After you confirm that the load job is successful, you can use to query the data of `table1` and `table2`: Query `table1`: ```SQL SELECT * FROM table1; ++-+-+ | id | name | score | ++-+-+ | 1 | Lily | 21 | | 2 | Rose | 22 | | 3 | Alice | 23 | | 4 | Julia | 24 | ++-+-+ 4 rows in set (0.01 sec) ``` Query `table2`: ```SQL SELECT * FROM table2; ++-+-+ | id | name | score | ++-+-+ | 5 | Tony | 25 | | 6 | Adam | 26 | | 7 | Allen | 27 | | 8 | Jacky | 28 | ++-+-+ 4 rows in set

sec) ``` Use the statement to query the results of one or more load jobs from the `loads` table in the `information_schema` database. This feature is supported from v3.1 onwards. Example 1: Query the results of load jobs executed on the `testdb` database. In the query statement, specify that a maximum of two results can be returned and the return results must be sorted by creation time (`CREATETIME`) in descending order. ```SQL SELECT * FROM information_schema.loads WHERE databasename = 'testdb' ORDER BY create_time DESC LIMIT 2\G ``` The following results are returned: ```SQL row * JOB_ID: 20686 LABEL: labelbrokerloadunqualifiedtest_83 DATABASENAME: testdb STATE: FINISHED PROGRESS: ETL:100%; LOAD:100% TYPE: BROKER PRIORITY: NORMAL SCAN_ROWS: 8 FILTERED_ROWS: 0 UNSELECTED_ROWS: 0 SINK_ROWS: 8 ETL_INFO: TASKINFO: resource:N/A; timeout(s):14400; maxfilter_ratio:1.0 CREATE_TIME: 2023-08-02 15:25:22 ETLSTARTTIME: 2023-08-02 15:25:24 ETLFINISHTIME: 2023-08-02 15:25:24 LOADSTARTTIME: 2023-08-02 15:25:24 LOADFINISHTIME: 2023-08-02 15:25:27 JOB_DETAILS: {"All backends":{"77fe760e-ec53-47f7-917d-be5528288c08":[10006],"0154f64e-e090-47b7-a4b2-92c2ece95f97":[10005]},"FileNumber":2,"FileSize":84,"InternalTableLoadBytes":252,"InternalTableLoadRows":8,"ScanBytes":84,"ScanRows":8,"TaskNumber":2,"Unfinished backends":{"77fe760e-ec53-47f7-917d-be5528288c08":[],"0154f64e-e090-47b7-a4b2-92c2ece95f97":[]}} ERROR_MSG: NULL TRACKING_URL: NULL TRACKING_SQL: NULL REJECTEDRECORDPATH: NULL row * JOB_ID: 20624 LABEL: labelbrokerloadunqualifiedtest_82 DATABASENAME: testdb STATE: FINISHED PROGRESS: ETL:100%; LOAD:100% TYPE: BROKER PRIORITY: NORMAL SCAN_ROWS: 12 FILTERED_ROWS: 4 UNSELECTED_ROWS: 0 SINK_ROWS: 8 ETL_INFO: TASKINFO: resource:N/A; timeout(s):14400; maxfilter_ratio:1.0 CREATE_TIME: 2023-08-02 15:23:29 ETLSTARTTIME: 2023-08-02 15:23:34 ETLFINISHTIME: 2023-08-02 15:23:34 LOADSTARTTIME: 2023-08-02 15:23:34 LOADFINISHTIME: 2023-08-02 15:23:34 JOB_DETAILS: {"All backends":{"78f78fc3-8509-451f-a0a2-c6b5db27dcb6":[10010],"a24aa357-f7de-4e49-9e09-e98463b5b53c":[10006]},"FileNumber":2,"FileSize":158,"InternalTableLoadBytes":333,"InternalTableLoadRows":8,"ScanBytes":158,"ScanRows":12,"TaskNumber":2,"Unfinished backends":{"78f78fc3-8509-451f-a0a2-c6b5db27dcb6":[],"a24aa357-f7de-4e49-9e09-e98463b5b53c":[]}} ERROR_MSG: NULL TRACKINGURL: http://172.26.195.69:8540/api/loaderrorlog?file=errorlog78f78fc38509451f_a0a2c6b5db27dcb7 TRACKINGSQL: select trackinglog from informationschema.loadtrackinglogs where jobid=20624 REJECTEDRECORDPATH: 172.26.95.92:/home/disk1/sr/be/storage/rejectedrecord/testdb/labelbrokerloadunqualifiedtest0728/6/404a20b1e4db4d278aa9af1e8d6d8bdc ``` Example 2: Query the result of the load job (whose label is `labelbrokerloadunqualifiedtest82`) executed on the `testdb` database: ```SQL SELECT * FROM information_schema.loads WHERE databasename = 'testdb' and label = 'labelbrokerloadunqualifiedtest_82'\G ``` The following result is returned: ```SQL row * JOB_ID: 20624 LABEL: labelbrokerloadunqualifiedtest_82 DATABASENAME: testdb STATE: FINISHED PROGRESS: ETL:100%; LOAD:100% TYPE: BROKER PRIORITY: NORMAL SCAN_ROWS: 12 FILTERED_ROWS: 4 UNSELECTED_ROWS: 0 SINK_ROWS: 8 ETL_INFO: TASKINFO: resource:N/A; timeout(s):14400; maxfilter_ratio:1.0 CREATE_TIME: 2023-08-02 15:23:29 ETLSTARTTIME: 2023-08-02 15:23:34 ETLFINISHTIME: 2023-08-02 15:23:34 LOADSTARTTIME: 2023-08-02 15:23:34 LOADFINISHTIME: 2023-08-02 15:23:34 JOB_DETAILS: {"All backends":{"78f78fc3-8509-451f-a0a2-c6b5db27dcb6":[10010],"a24aa357-f7de-4e49-9e09-e98463b5b53c":[10006]},"FileNumber":2,"FileSize":158,"InternalTableLoadBytes":333,"InternalTableLoadRows":8,"ScanBytes":158,"ScanRows":12,"TaskNumber":2,"Unfinished backends":{"78f78fc3-8509-451f-a0a2-c6b5db27dcb6":[],"a24aa357-f7de-4e49-9e09-e98463b5b53c":[]}} ERROR_MSG: NULL TRACKINGURL: http://172.26.195.69:8540/api/loaderrorlog?file=errorlog78f78fc38509451f_a0a2c6b5db27dcb7 TRACKINGSQL: select trackinglog from informationschema.loadtrackinglogs where jobid=20624 REJECTEDRECORDPATH: 172.26.95.92:/home/disk1/sr/be/storage/rejectedrecord/testdb/labelbrokerloadunqualifiedtest0728/6/404a20b1e4db4d278aa9af1e8d6d8bdc ``` For information about the fields in the return results, see . When a load job is not in the CANCELLED or FINISHED stage, you can use the statement to cancel the job. For example, you can execute the following statement to cancel a load job, whose label is `label1`, in the database `test_db`: ```SQL CANCEL LOAD FROM test_db WHERE LABEL = "label1"; ``` A Broker Load job can be split into one or more tasks that concurrently run. The tasks within a load job are run within a single transaction. They must all succeed or fail. StarRocks splits each load job based on how you declare `data_desc` in the `LOAD` statement: If you declare multiple `data_desc` parameters, each of which specifies a distinct table, a task is generated to load the data of each table. If you declare multiple `data_desc` parameters, each of which specifies a distinct partition for the same table, a task is generated to load the data of each partition. Additionally, each task can be further split into one or more instances, which are evenly distributed to and concurrently run on the BEs or CNs of your StarRocks cluster. StarRocks splits each task based on the FE parameter and the number of BE or CN nodes. You can use the following formula to calculate the number of instances in an individual task: Number of instances in an individual task = min(Amount of data to be loaded by an individual task/`min_bytes_per_broker_scanner`, Number of BE/CN nodes) In most cases, only one `data_desc` is declared for each load job, each load job is split into only one task, and the task is split into the same number of instances as the number of BE or CN