Repository: pingcap/tidb
Pull request #60384: executor: tiny optimize import into from select performance
Author: login: crazycs520
Labels: name: size/L
description: Denotes a PR that changes 100-499 lines, ignoring generated files.
name: release-note-none
description: Denotes a PR that doesn't merit a release note.
name: approved
name: lgtm
Created: 2025-04-02T17:31:07Z | Merged: 2025-04-22T11:23:26Z

<!--

Thank you for contributing to TiDB!

PR Title Format:
1. pkg [, pkg2, pkg3]: what's changed
2. *: what's changed

-->

### What problem does this PR solve?
<!--

Please create an issue first to describe the problem.

There MUST be one line starting with "Issue Number:  " and
linking the relevant issues via the "close" or "ref".

For more info, check https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/contribute-code.html#referring-to-an-issue.

-->

Issue Number: ref #60377

Problem Summary: tiny optimize import into from select performance

### What changed and how does it work?

Tiny fix for better pipeline.

Manual test:

1. prepare:

```sh
sysbench --config-file=sysbench.conf oltp_point_select --tables=2 --table-size=10000 --threads=2 prepare

CREATE TABLE `t` (
  `id` int(11) NOT NULL,
  `k` int(11) NOT NULL DEFAULT '0',
  `c` char(120) NOT NULL DEFAULT '',
  `pad` char(60) NOT NULL DEFAULT '',
  `id2` int(11) NOT NULL,
  `k2` int(11) NOT NULL DEFAULT '0',
  `c2` char(120) NOT NULL DEFAULT '',
  `pad2` char(60) NOT NULL DEFAULT '',
  KEY `k_1` (`k`)
);
```


#### Before:

```master
> import into t from select * from sbtest1 join sbtest2 where sbtest2.id <=2000 with thread=16,DISABLE_PRECHECK;
Query OK, 20000000 rows affected
Time: 94.425s
```
related tidb log:

```log
[2025/04/03 01:24:12.759 +08:00] [INFO] [chunk_process.go:334] ["process chunk completed"] [table=t] [import-id=431840f5-f697-4b3b-8093-e8dcb329d77a] [key=import-from-select] [readDur=43.395559463s] [encodeDur=9.013400878s] [checksum="{cksum=1539544195137435216,size=563438023,kvs=2499458}"] [deliverDur=1.608033096s] [type=query] [takeTime=53.673599639s] []
```

**The `readDur` has `43.39s`.**

```log
[readDur=43.395559463s] [encodeDur=9.013400878s] [checksum="{cksum=1539544195137435216,size=563438023,kvs=2499458}"] [deliverDur=1.608033096s] [type=query] [takeTime=53.673599639s] 
```


#### This PR:

```sql
> import into t from select * from sbtest1 join sbtest2 where sbtest2.id <=2000 with thread=16,DISABLE_PRECHECK;
Query OK, 20000000 rows affected
Time: 59.718s
```
related tidb log:

```log
[2025/04/03 01:26:32.376 +08:00] [INFO] [chunk_process.go:361] ["process chunk completed"] [table=t] [import-id=6bee0397-56b5-4e8b-9383-43f5e8593f22] [key=import-from-select] [readDur=6.501533276s] [encodeDur=9.75518629s] [checksum="{cksum=4058046773982146946,size=564270116,kvs=2503168}"] [deliverDur=1.788105169s] [type=query] [takeTime=18.361915694s] []
```

**The `readDur` only has `6.5s`.**

```log
[readDur=6.501533276s] [encodeDur=9.75518629s] [checksum="{cksum=4058046773982146946,size=564270116,kvs=2503168}"] [deliverDur=1.788105169s] [type=query] [takeTime=18.361915694s]
```

### Check List

Tests <!-- At least one of them must be included. -->

- [ ] Unit test
- [ ] Integration test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No need to test
  > - [ ] I checked and no code files have been changed.
  > <!-- Or your custom  "No need to test" reasons -->

Side effects

- [ ] Performance regression: Consumes more CPU
- [ ] Performance regression: Consumes more Memory
- [ ] Breaking backward compatibility

Documentation

- [ ] Affects user behaviors
- [ ] Contains syntax changes
- [ ] Contains variable changes
- [ ] Contains experimental features
- [ ] Changes MySQL compatibility

### Release note

<!-- compatibility change, improvement, bugfix, and new feature need a release note -->

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

```release-note
None
```



Diff:
diff --git a/pkg/executor/import_into.go b/pkg/executor/import_into.go
index 170d71f772e68..953aaf65ebd61 100644
--- a/pkg/executor/import_into.go
+++ b/pkg/executor/import_into.go
@@ -276,8 +276,8 @@ func (e *ImportIntoExec) importFromSelect(ctx context.Context) error {
 			logutil.Logger(ctx).Error("close importer failed", zap.Error(err))
 		}
 	}()
-	selectedRowCh := make(chan importer.QueryRow)
-	ti.SetSelectedRowCh(selectedRowCh)
+	selectedChunkCh := make(chan importer.QueryChunk, 1)
+	ti.SetSelectedChunkCh(selectedChunkCh)
 
 	var importResult *importer.JobImportResult
 	eg, egCtx := errgroup.WithContext(ctx)
@@ -287,13 +287,14 @@ func (e *ImportIntoExec) importFromSelect(ctx context.Context) error {
 		return err
 	})
 	eg.Go(func() error {
-		defer close(selectedRowCh)
+		defer close(selectedChunkCh)
 		fields := exec.RetTypes(e.selectExec)
 		var idAllocator int64
+		chkSize := e.selectExec.InitCap()
+		maxChkSize := e.selectExec.MaxChunkSize()
 		for {
 			// rows will be consumed concurrently, we cannot use chunk pool in session ctx.
-			chk := exec.NewFirstChunk(e.selectExec)
-			iter := chunk.NewIterator4Chunk(chk)
+			chk := chunk.New(e.selectExec.RetFieldTypes(), chkSize, maxChkSize)
 			err := exec.Next(egCtx, e.selectExec, chk)
 			if err != nil {
 				return err
@@ -301,16 +302,19 @@ func (e *ImportIntoExec) importFromSelect(ctx context.Context) error {
 			if chk.NumRows() == 0 {
 				break
 			}
-			for innerChunkRow := iter.Begin(); innerChunkRow != iter.End(); innerChunkRow = iter.Next() {
-				idAllocator++
-				select {
-				case selectedRowCh <- importer.QueryRow{
-					ID:   idAllocator,
-					Data: innerChunkRow.GetDatumRow(fields),
-				}:
-				case <-egCtx.Done():
-					return egCtx.Err()
-				}
+			select {
+			case selectedChunkCh <- importer.QueryChunk{
+				Fields:      fields,
+				Chk:         chk,
+				RowIDOffset: idAllocator,
+			}:
+				idAllocator += int64(chk.NumRows())
+			case <-egCtx.Done():
+				return egCtx.Err()
+			}
+			if chkSize < maxChkSize {
+				chkSize = chkSize * 2
+				chkSize = min(chkSize, maxChkSize)
 			}
 		}
 		return nil
diff --git a/pkg/executor/importer/BUILD.bazel b/pkg/executor/importer/BUILD.bazel
index 13760bcbb9ee4..1b1ea262c73a2 100644
--- a/pkg/executor/importer/BUILD.bazel
+++ b/pkg/executor/importer/BUILD.bazel
@@ -141,6 +141,7 @@ go_test(
         "//pkg/types",
         "//pkg/util",
         "//pkg/util/cdcutil",
+        "//pkg/util/chunk",
         "//pkg/util/dbterror/exeerrors",
         "//pkg/util/dbterror/plannererrors",
         "//pkg/util/etcd",
diff --git a/pkg/executor/importer/chunk_process.go b/pkg/executor/importer/chunk_process.go
index 140cfbfe2e3c2..646c88311d731 100644
--- a/pkg/executor/importer/chunk_process.go
+++ b/pkg/executor/importer/chunk_process.go
@@ -34,6 +34,7 @@ import (
 	"github.com/pingcap/tidb/pkg/tablecodec"
 	"github.com/pingcap/tidb/pkg/types"
 	"github.com/pingcap/tidb/pkg/util"
+	"github.com/pingcap/tidb/pkg/util/chunk"
 	"github.com/pingcap/tidb/pkg/util/syncutil"
 	"github.com/prometheus/client_golang/prometheus"
 	"go.uber.org/zap"
@@ -91,27 +92,47 @@ func parserEncodeReader(parser mydump.Parser, endOffset int64, filename string)
 	}
 }
 
-// queryRowEncodeReader wraps a QueryRow channel as a encodeReaderFn.
-func queryRowEncodeReader(rowCh <-chan QueryRow) encodeReaderFn {
-	return func(ctx context.Context) (data rowToEncode, closed bool, err error) {
+type queryChunkEncodeReader struct {
+	chunkCh <-chan QueryChunk
+	currChk QueryChunk
+	cursor  int
+	numRows int
+}
+
+func (r *queryChunkEncodeReader) readRow(ctx context.Context) (data rowToEncode, closed bool, err error) {
+	if r.currChk.Chk == nil || r.cursor >= r.numRows {
 		select {
 		case <-ctx.Done():
 			err = ctx.Err()
 			return
-		case row, ok := <-rowCh:
+		case currChk, ok := <-r.chunkCh:
 			if !ok {
 				closed = true
 				return
 			}
-			data = rowToEncode{
-				row:       row.Data,
-				rowID:     row.ID,
-				endOffset: -1,
-				resetFn:   func() {},
-			}
-			return
+			r.currChk = currChk
+			r.cursor = 0
+			r.numRows = r.currChk.Chk.NumRows()
 		}
 	}
+
+	row := r.currChk.Chk.GetRow(r.cursor)
+	r.cursor++
+	data = rowToEncode{
+		row:       row.GetDatumRow(r.currChk.Fields),
+		rowID:     r.currChk.RowIDOffset + int64(r.cursor),
+		endOffset: -1,
+		resetFn:   func() {},
+	}
+	return
+}
+
+// queryRowEncodeReader wraps a queryChunkEncodeReader as a encodeReaderFn.
+func queryRowEncodeReader(chunkCh <-chan QueryChunk) encodeReaderFn {
+	reader := queryChunkEncodeReader{chunkCh: chunkCh}
+	return func(ctx context.Context) (data rowToEncode, closed bool, err error) {
+		return reader.readRow(ctx)
+	}
 }
 
 type encodedKVGroupBatch struct {
@@ -493,14 +514,15 @@ func (p *dataDeliver) summaryFields() []zap.Field {
 	}
 }
 
-// QueryRow is a row from query result.
-type QueryRow struct {
-	ID   int64
-	Data []types.Datum
+// QueryChunk is a chunk from query result.
+type QueryChunk struct {
+	Fields      []*types.FieldType
+	Chk         *chunk.Chunk
+	RowIDOffset int64
 }
 
 func newQueryChunkProcessor(
-	rowCh chan QueryRow,
+	chunkCh chan QueryChunk,
 	encoder KVEncoder,
 	keyspace []byte,
 	logger *zap.Logger,
@@ -523,7 +545,7 @@ func newQueryChunkProcessor(
 		deliver:    deliver,
 		enc: newChunkEncoder(
 			chunkName,
-			queryRowEncodeReader(rowCh),
+			queryRowEncodeReader(chunkCh),
 			-1,
 			deliver.sendEncodedData,
 			chunkLogger,
diff --git a/pkg/executor/importer/engine_process.go b/pkg/executor/importer/engine_process.go
index 8d00be701c367..0c834a948f3a0 100644
--- a/pkg/executor/importer/engine_process.go
+++ b/pkg/executor/importer/engine_process.go
@@ -106,7 +106,7 @@ func ProcessChunkWithWriter(
 		)
 	case DataSourceTypeQuery:
 		cp = newQueryChunkProcessor(
-			tableImporter.rowCh, encoder, tableImporter.GetKeySpace(), logger,
+			tableImporter.chunkCh, encoder, tableImporter.GetKeySpace(), logger,
 			tableImporter.diskQuotaLock, dataWriter, indexWriter, groupChecksum,
 		)
 	}
diff --git a/pkg/executor/importer/importer_testkit_test.go b/pkg/executor/importer/importer_testkit_test.go
index 4d11dd15aa694..7bcfdc51dd249 100644
--- a/pkg/executor/importer/importer_testkit_test.go
+++ b/pkg/executor/importer/importer_testkit_test.go
@@ -38,6 +38,7 @@ import (
 	"github.com/pingcap/tidb/pkg/meta/autoid"
 	"github.com/pingcap/tidb/pkg/meta/model"
 	"github.com/pingcap/tidb/pkg/parser/ast"
+	"github.com/pingcap/tidb/pkg/parser/mysql"
 	plannercore "github.com/pingcap/tidb/pkg/planner/core"
 	"github.com/pingcap/tidb/pkg/planner/core/base"
 	"github.com/pingcap/tidb/pkg/planner/core/resolve"
@@ -46,6 +47,7 @@ import (
 	"github.com/pingcap/tidb/pkg/testkit"
 	"github.com/pingcap/tidb/pkg/testkit/testfailpoint"
 	"github.com/pingcap/tidb/pkg/types"
+	"github.com/pingcap/tidb/pkg/util/chunk"
 	"github.com/pingcap/tidb/pkg/util/dbterror/exeerrors"
 	"github.com/pingcap/tidb/pkg/util/logutil"
 	"github.com/stretchr/testify/require"
@@ -331,19 +333,35 @@ func TestProcessChunkWith(t *testing.T) {
 			ti.LoadDataController.Close()
 			ti.Backend().CloseEngineMgr()
 		}()
-		rowsCh := make(chan importer.QueryRow, 3)
-		for i := 1; i <= 3; i++ {
-			rowsCh <- importer.QueryRow{
-				ID: int64(i),
-				Data: []types.Datum{
-					types.NewIntDatum(int64((i-1)*3 + 1)),
-					types.NewIntDatum(int64((i-1)*3 + 2)),
-					types.NewIntDatum(int64((i-1)*3 + 3)),
-				},
-			}
+		chkCh := make(chan importer.QueryChunk, 3)
+		fields := make([]*types.FieldType, 0, 3)
+		for i := 0; i < 3; i++ {
+			fields = append(fields, types.NewFieldType(mysql.TypeLong))
 		}
-		close(rowsCh)
-		ti.SetSelectedRowCh(rowsCh)
+		chk := chunk.New(fields, 2, 2)
+		for i := 1; i <= 2; i++ {
+			chk.AppendInt64(0, int64((i-1)*3+1))
+			chk.AppendInt64(1, int64((i-1)*3+2))
+			chk.AppendInt64(2, int64((i-1)*3+3))
+		}
+		chkCh <- importer.QueryChunk{
+			Fields:      fields,
+			Chk:         chk,
+			RowIDOffset: 0,
+		}
+		chk = chunk.New(fields, 1, 1)
+		for i := 3; i <= 3; i++ {
+			chk.AppendInt64(0, int64((i-1)*3+1))
+			chk.AppendInt64(1, int64((i-1)*3+2))
+			chk.AppendInt64(2, int64((i-1)*3+3))
+		}
+		chkCh <- importer.QueryChunk{
+			Fields:      fields,
+			Chk:         chk,
+			RowIDOffset: 2,
+		}
+		close(chkCh)
+		ti.SetSelectedChunkCh(chkCh)
 		kvWriter := mock.NewMockEngineWriter(ctrl)
 		kvWriter.EXPECT().AppendRows(gomock.Any(), gomock.Any(), gomock.Any()).Return(nil).AnyTimes()
 		checksum := verify.NewKVGroupChecksumWithKeyspace(keyspace)
diff --git a/pkg/executor/importer/table_import.go b/pkg/executor/importer/table_import.go
index cfed495712a61..67974496c36df 100644
--- a/pkg/executor/importer/table_import.go
+++ b/pkg/executor/importer/table_import.go
@@ -243,7 +243,7 @@ type TableImporter struct {
 	diskQuota       int64
 	diskQuotaLock   *syncutil.RWMutex
 
-	rowCh chan QueryRow
+	chunkCh chan QueryChunk
 }
 
 // NewTableImporterForTest creates a new table importer for test.
@@ -614,9 +614,9 @@ func (ti *TableImporter) CheckDiskQuota(ctx context.Context) {
 	}
 }
 
-// SetSelectedRowCh sets the channel to receive selected rows.
-func (ti *TableImporter) SetSelectedRowCh(ch chan QueryRow) {
-	ti.rowCh = ch
+// SetSelectedChunkCh sets the channel to receive selected rows.
+func (ti *TableImporter) SetSelectedChunkCh(ch chan QueryChunk) {
+	ti.chunkCh = ch
 }
 
 func (ti *TableImporter) closeAndCleanupEngine(engine *backend.OpenedEngine) {
diff --git a/pkg/executor/importer/table_import_testkit_test.go b/pkg/executor/importer/table_import_testkit_test.go
index 020f264a34d17..2459b64b71648 100644
--- a/pkg/executor/importer/table_import_testkit_test.go
+++ b/pkg/executor/importer/table_import_testkit_test.go
@@ -26,12 +26,14 @@ import (
 	"github.com/pingcap/tidb/pkg/lightning/backend/local"
 	"github.com/pingcap/tidb/pkg/meta/model"
 	"github.com/pingcap/tidb/pkg/parser/ast"
+	"github.com/pingcap/tidb/pkg/parser/mysql"
 	plannercore "github.com/pingcap/tidb/pkg/planner/core"
 	"github.com/pingcap/tidb/pkg/planner/core/resolve"
 	"github.com/pingcap/tidb/pkg/session"
 	"github.com/pingcap/tidb/pkg/testkit"
 	"github.com/pingcap/tidb/pkg/types"
 	"github.com/pingcap/tidb/pkg/util"
+	"github.com/pingcap/tidb/pkg/util/chunk"
 	"github.com/stretchr/testify/require"
 	"github.com/tikv/client-go/v2/tikv"
 )
@@ -105,18 +107,27 @@ func TestImportFromSelectCleanup(t *testing.T) {
 		&storeHelper{kvStore: store},
 	)
 	require.NoError(t, err)
-	ch := make(chan importer.QueryRow)
-	ti.SetSelectedRowCh(ch)
+	ch := make(chan importer.QueryChunk)
+	ti.SetSelectedChunkCh(ch)
 	var wg util.WaitGroupWrapper
 	wg.Run(func() {
 		defer close(ch)
-		for i := 1; i <= 3; i++ {
-			ch <- importer.QueryRow{
-				ID: int64(i),
-				Data: []types.Datum{
-					types.NewIntDatum(int64(i)),
-				},
-			}
+		fields := make([]*types.FieldType, 0, 3)
+		fields = append(fields, types.NewFieldType(mysql.TypeLong))
+		chk := chunk.New(fields, 2, 2)
+		chk.AppendInt64(0, int64(1))
+		chk.AppendInt64(0, int64(2))
+		ch <- importer.QueryChunk{
+			Fields:      fields,
+			Chk:         chk,
+			RowIDOffset: 0,
+		}
+		chk = chunk.New(fields, 1, 1)
+		chk.AppendInt64(0, int64(3))
+		ch <- importer.QueryChunk{
+			Fields:      fields,
+			Chk:         chk,
+			RowIDOffset: 2,
 		}
 	})
 	_, err = ti.ImportSelectedRows(ctx, tk.Session())


Reviews:
user: login: D3Hunter
body: will `send by chunk` make performance worse if the data-set is small?
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-03T02:27:16Z
user: login: D3Hunter
body: rest lgtm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-17T07:45:23Z
user: login: crazycs520
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-21T14:12:24Z
user: login: crazycs520
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-21T14:12:29Z
user: login: D3Hunter
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T04:57:24Z
user: login: lance6716
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T06:03:31Z

Review comments:
diff_hunk: @@ -91,27 +92,47 @@ func parserEncodeReader(parser mydump.Parser, endOffset int64, filename string)
 	}
 }
 
-// queryRowEncodeReader wraps a QueryRow channel as a encodeReaderFn.
-func queryRowEncodeReader(rowCh <-chan QueryRow) encodeReaderFn {
-	return func(ctx context.Context) (data rowToEncode, closed bool, err error) {
+type queryChunkEncodeReader struct {
+	chunkCh  <-chan QueryChunk
+	queryChk QueryChunk
path: pkg/executor/importer/chunk_process.go
user: login: D3Hunter
body: ```suggestion
	currChk QueryChunk
```
created_at: 2025-04-17T07:37:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -493,14 +514,15 @@ func (p *dataDeliver) summaryFields() []zap.Field {
 	}
 }
 
-// QueryRow is a row from query result.
-type QueryRow struct {
-	ID   int64
-	Data []types.Datum
+// QueryChunk is a chunk from query result.
+type QueryChunk struct {
+	Fields []*types.FieldType
+	Chk    *chunk.Chunk
+	Offset int64
path: pkg/executor/importer/chunk_process.go
user: login: D3Hunter
body: ```suggestion
	RowIDOffset int64
```
created_at: 2025-04-17T07:39:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -91,27 +92,47 @@ func parserEncodeReader(parser mydump.Parser, endOffset int64, filename string)
 	}
 }
 
-// queryRowEncodeReader wraps a QueryRow channel as a encodeReaderFn.
-func queryRowEncodeReader(rowCh <-chan QueryRow) encodeReaderFn {
-	return func(ctx context.Context) (data rowToEncode, closed bool, err error) {
+type queryChunkEncodeReader struct {
+	chunkCh  <-chan QueryChunk
+	queryChk QueryChunk
path: pkg/executor/importer/chunk_process.go
user: login: crazycs520
body: done.
created_at: 2025-04-21T14:12:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -493,14 +514,15 @@ func (p *dataDeliver) summaryFields() []zap.Field {
 	}
 }
 
-// QueryRow is a row from query result.
-type QueryRow struct {
-	ID   int64
-	Data []types.Datum
+// QueryChunk is a chunk from query result.
+type QueryChunk struct {
+	Fields []*types.FieldType
+	Chk    *chunk.Chunk
+	Offset int64
path: pkg/executor/importer/chunk_process.go
user: login: crazycs520
body: done
created_at: 2025-04-21T14:12:29Z
author_association: CONTRIBUTOR

PR comments:
user: login: sre-bot
created_at: 2025-04-02T17:31:14Z
body: [![CLA assistant check](https://cla.pingcap.net/pull/badge/signed)](https://cla.pingcap.net/pingcap/tidb?pullRequest=60384) <br/>All committers have signed the CLA.
author_association: CONTRIBUTOR
user: login: tiprow[bot]
created_at: 2025-04-02T17:31:24Z
body: Hi @crazycs520. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with `/ok-to-test` in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using `/test all`.

I understand the commands that are listed [here](https://go.k8s.io/bot-commands?repo=pingcap%2Ftidb).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>

author_association: NONE
user: login: codecov[bot]
created_at: 2025-04-02T17:58:47Z
body: ## [Codecov](https://app.codecov.io/gh/pingcap/tidb/pull/60384?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) Report
Attention: Patch coverage is `62.16216%` with `14 lines` in your changes missing coverage. Please review.
> Project coverage is 74.7785%. Comparing base [(`3cdb05c`)](https://app.codecov.io/gh/pingcap/tidb/commit/3cdb05c4a6dd97d8127e17b6058aea2ac38f3d45?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) to head [(`50cbc5e`)](https://app.codecov.io/gh/pingcap/tidb/commit/50cbc5e8972169cfc68b663c474986fa84bbe26f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap).
> Report is 4 commits behind head on master.

<details><summary>Additional details and impacted files</summary>


```diff
@@               Coverage Diff                @@
##             master     #60384        +/-   ##
================================================
+ Coverage   73.1122%   74.7785%   +1.6662%     
================================================
  Files          1716       1732        +16     
  Lines        475637     475984       +347     
================================================
+ Hits         347749     355934      +8185     
+ Misses       106491      98105      -8386     
- Partials      21397      21945       +548     
```

| [Flag](https://app.codecov.io/gh/pingcap/tidb/pull/60384/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | |
|---|---|---|
| [integration](https://app.codecov.io/gh/pingcap/tidb/pull/60384/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `46.0874% <0.0000%> (?)` | |
| [unit](https://app.codecov.io/gh/pingcap/tidb/pull/60384/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `72.5020% <62.1621%> (+0.1760%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Components](https://app.codecov.io/gh/pingcap/tidb/pull/60384/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | |
|---|---|---|
| [dumpling](https://app.codecov.io/gh/pingcap/tidb/pull/60384/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `52.6553% <ø> (ø)` | |
| [parser](https://app.codecov.io/gh/pingcap/tidb/pull/60384/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `∅ <ø> (∅)` | |
| [br](https://app.codecov.io/gh/pingcap/tidb/pull/60384/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `62.1958% <ø> (+14.7662%)` | :arrow_up: |

</details>
<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
author_association: NONE
user: login: crazycs520
created_at: 2025-04-07T05:44:17Z
body: > will `send by chunk` make performance worse if the data-set is small?

I'm not sure what small data-set you mean, 10000 rows? If it's too small, I don't feel like it's making much difference.

IMO, `import into` is usually used in cases where the amount of data is relatively large.
author_association: CONTRIBUTOR
user: login: crazycs520
created_at: 2025-04-17T01:28:33Z
body: /test check-dev2
author_association: CONTRIBUTOR
user: login: tiprow[bot]
created_at: 2025-04-17T01:28:56Z
body: @crazycs520: Cannot trigger testing until a trusted user reviews the PR and leaves an `/ok-to-test` message.

<details>

In response to [this](https://github.com/pingcap/tidb/pull/60384#issuecomment-2811472485):

>/test check-dev2


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>
author_association: NONE
user: login: ti-chi-bot[bot]
created_at: 2025-04-22T06:03:37Z
body: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/pingcap/tidb/pull/60384#pullrequestreview-2782758337" title="Approved">D3Hunter</a>*, *<a href="https://github.com/pingcap/tidb/pull/60384#pullrequestreview-2782851225" title="Approved">lance6716</a>*

The full list of commands accepted by this bot can be found [here](https://prow.tidb.net/command-help?repo=pingcap%2Ftidb).

The pull request process is described [here](https://book.prow.tidb.net/#/workflows/pr)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/pingcap/tidb/blob/master/OWNERS)~~ [D3Hunter,lance6716]
- ~~[pkg/executor/importer/OWNERS](https://github.com/pingcap/tidb/blob/master/pkg/executor/importer/OWNERS)~~ [D3Hunter]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
author_association: NONE
user: login: lance6716
created_at: 2025-04-22T06:03:41Z
body: /retest
author_association: CONTRIBUTOR
user: login: ti-chi-bot[bot]
created_at: 2025-04-22T06:03:41Z
body: [LGTM Timeline notifier]
---
Timeline:

- `2025-04-22 04:57:33.24573833 +0000 UTC m=+331597.057528712`: :ballot_box_with_check: agreed by [D3Hunter](https://github.com/D3Hunter).
- `2025-04-22 06:03:41.177123156 +0000 UTC m=+335564.988913537`: :ballot_box_with_check: agreed by [lance6716](https://github.com/lance6716).
author_association: NONE
user: login: crazycs520
created_at: 2025-04-22T08:24:29Z
body: /retest-required
author_association: CONTRIBUTOR
user: login: crazycs520
created_at: 2025-04-22T10:14:44Z
body: /retest-required
author_association: CONTRIBUTOR