Repository: pingcap/tidb (Go)

Issue:
Lost connection to MySQL server during query
## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

<!-- a step by step guide for reproducing the bug. -->

use test;
source [test.txt](https://github.com/pingcap/tidb/files/5281854/test.txt);
SELECT /*+ inl_hash_join(t1) */ t1.pk, t2.pk from table_290_utf8_undef t1 right join table_290_utf8_undef t2 on t1. `col_char20_key_signed` = t2. `col_set_key_signed` and t1. `col_bool_key_signed` = t2. `col_enum_key_signed` where t1. `col_smallint_key_unsigned` != -969 order by t1.pk, t2.pk;

tidb stack info
```
panic: runtime error: index out of range [4194303] with length 26 [recovered]
	panic: send on closed channel

goroutine 146332 [running]:
github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).handleHashJoinInnerWorkerPanic(0xc0013c0600, 0x2e84d60, 0xc001b933a0)
	/root/go/src/github.com/pingcap/tidb/executor/index_lookup_hash_join.go:572 +0x10a
github.com/pingcap/tidb/util.WithRecovery.func1(0xc001ec8fd0)
	/root/go/src/github.com/pingcap/tidb/util/misc.go:88 +0x324
panic(0x2e84d60, 0xc001b933a0)
	/usr/local/go/src/runtime/panic.go:969 +0x166
github.com/pingcap/tidb/util/codec.encodeHashChunkRowIdx(0xc0001eb600, 0xc000c9a4b0, 0x0, 0xc0011b5cb0, 0x1, 0xc001bc3800, 0xc0011dee10, 0x107c7ff, 0x2cad100, 0xc0011dee10, ...)
	/root/go/src/github.com/pingcap/tidb/util/codec/codec.go:367 +0xc10
github.com/pingcap/tidb/util/codec.HashChunkRow(0xc0001eb600, 0x7fd4e42ec0f8, 0xc0015227e0, 0xc000c9a4b0, 0x0, 0xc00190e080, 0x3, 0x3, 0xc001e3da80, 0x1, ...)
	/root/go/src/github.com/pingcap/tidb/util/codec/codec.go:660 +0x110
github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).buildHashTableForOuterResult(0xc0013c0600, 0x358dc20, 0xc001b8fc80, 0xc001ea78c0, 0x35a8520, 0xc0015227e0)
	/root/go/src/github.com/pingcap/tidb/executor/index_lookup_hash_join.go:548 +0x3b1
github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).handleTask.func2()
	/root/go/src/github.com/pingcap/tidb/executor/index_lookup_hash_join.go:591 +0x57
github.com/pingcap/tidb/util.WithRecovery(0xc001c48c80, 0xc001ec8fd0)
	/root/go/src/github.com/pingcap/tidb/util/misc.go:96 +0x4f
created by github.com/pingcap/tidb/executor.(*indexHashJoinInnerWorker).handleTask
	/root/go/src/github.com/pingcap/tidb/executor/index_lookup_hash_join.go:591 +0x1bb
```
### 2. What did you expect to see? (Required)
Don't have runtime error

### 3. What did you see instead (Required)
infinite loop query sql 
```
mysql> SELECT /*+ inl_hash_join(t1) */ t1.pk, t2.pk from table_290_utf8_undef t1 right join table_290_utf8_undef t2 on t1. `col_char20_key_signed` = t2. `col_set_key_signed` and t1. `col_bool_key_signed` = t2. `col_enum_key_signed` where t1. `col_smallint_key_unsigned` != -969 order by t1.pk, t2.pk;
ERROR 1105 (HY000): runtime error: index out of range [65535] with length 26
mysql> SELECT /*+ inl_hash_join(t1) */ t1.pk, t2.pk from table_290_utf8_undef t1 right join table_290_utf8_undef t2 on t1. `col_char20_key_signed` = t2. `col_set_key_signed` and t1. `col_bool_key_signed` = t2. `col_enum_key_signed` where t1. `col_smallint_key_unsigned` != -969 order by t1.pk, t2.pk;
ERROR 2013 (HY000): Lost connection to MySQL server during query
```
### 4. What is your TiDB version? (Required)

<!-- Paste the output of SELECT tidb_version() -->
master 32d86e26c and release-4.0 ed939f3f1



Discussion:
@ChenPeng2013 I was trying to reproduce this issue, however when I ran `source test.txt`, it says

```
mysql root@localhost:test> source test.txt;
'utf-8' codec can't decode byte 0xff in position 4432: invalid start byte
```

On TiDB, but it works for MySQL.

Is this a bug or am I misconfiguring something?
minor case:
``` sql
CREATE TABLE `t` (   `a` set('a','b','c','d','e','f','g','h','i','j') DEFAULT NULL );
insert into t values('i'), ('j');
 CREATE TABLE `s` (
  `a` char(1) DEFAULT NULL,
  KEY `a` (`a`)
);
insert into s values('i','j');
select /*+ inl_hash_join(s)*/ t.a from t left join s on t.a = s.a;
```
## Please edit this comment or add a new comment to complete the following information

### Not a bug

1. Remove the 'type/bug' label
2. Add notes to indicate why it is not a bug

### Duplicate bug

1. Add the 'type/duplicate' label
2. Add the link to the original bug

### Bug

Note: Make Sure that 'component', and 'severity' labels are added
Example for how to fill out the template: https://github.com/pingcap/tidb/issues/20100

#### 1. Root Cause Analysis (RCA) (optional) 
<!-- Write down the reason why this bug occurs -->

#### 2. Symptom (optional)

<!-- What will the user see when this bug occurs. The error message may be in the terminal, log or monitoring -->

#### 3. All Trigger Conditions (optional)

<!-- All the user scenarios that may trigger this bug -->

#### 4. Workaround (optional)

#### 5. Affected versions

<!--
In the format of [start_version:end_version], multiple version ranges are
accepted. If the bug only affects the unreleased version, please input:
"unreleased". For example:

Notes:
  1. Do not use any white spaces in '[]'.
  2. The range in '[]' is a closed interval
  3. The version format is `v$Major.$Minor.$Patch`, the $Majoy and $Minor
     number in a version range should be the same. [v3.0.1:v3.1.2] is
     invalid because the $Minor number of the version range is different.

Example 1: [v3.0.1:v3.0.5], [v4.0.1:v4.0.5]
Example 2: unreleased
-->

#### 6. Fixed versions

<!--
The first released version that contains this fix in each minor version. If the bug's affected version has been released, the fixed version should be a detailed version number; If the bug doesn't affect any released version, the fixed version can be "master";  

Example 1: v3.0.13, v4.0.5
Example 2: master
-->



Patch:
diff --git a/util/codec/codec.go b/util/codec/codec.go
index 38a99b53b85dc..8ede0bb67bf27 100644
--- a/util/codec/codec.go
+++ b/util/codec/codec.go
@@ -363,9 +363,11 @@ func encodeHashChunkRowIdx(sc *stmtctx.StatementContext, row chunk.Row, tp *type
 		b = ConvertByCollation(hack.Slice(str), tp)
 	case mysql.TypeSet:
 		flag = compactBytesFlag
-		v := uint64(row.GetSet(idx).ToNumber())
-		str := tp.Elems[v-1]
-		b = ConvertByCollation(hack.Slice(str), tp)
+		s, err := types.ParseSetValue(tp.Elems, row.GetSet(idx).Value)
+		if err != nil {
+			return 0, nil, err
+		}
+		b = ConvertByCollation(hack.Slice(s.Name), tp)
 	case mysql.TypeBit:
 		// We don't need to handle errors here since the literal is ensured to be able to store in uint64 in convertToMysqlBit.
 		flag = uvarintFlag
@@ -588,9 +590,11 @@ func HashChunkSelected(sc *stmtctx.StatementContext, h []hash.Hash64, chk *chunk
 				isNull[i] = !ignoreNull
 			} else {
 				buf[0] = compactBytesFlag
-				v := uint64(column.GetSet(i).ToNumber())
-				str := tp.Elems[v-1]
-				b = ConvertByCollation(hack.Slice(str), tp)
+				s, err := types.ParseSetValue(tp.Elems, column.GetSet(i).Value)
+				if err != nil {
+					return err
+				}
+				b = ConvertByCollation(hack.Slice(s.Name), tp)
 			}
 
 			// As the golang doc described, `Hash.Write` never returns an error.


Test patch:
diff --git a/executor/join_test.go b/executor/join_test.go
index 09dad2121a292..e83c019d0ba09 100644
--- a/executor/join_test.go
+++ b/executor/join_test.go
@@ -2473,3 +2473,16 @@ func (s *testSuiteJoinSerial) TestIssue20779(c *C) {
 	_, err = session.GetRows4Test(context.Background(), nil, rs)
 	c.Assert(err.Error(), Matches, "testIssue20779")
 }
+
+func (s *testSuiteJoinSerial) TestIssue20219(c *C) {
+	tk := testkit.NewTestKitWithInit(c, s.store)
+	tk.MustExec("drop table if exists t,s ")
+	tk.MustExec("CREATE TABLE `t` (   `a` set('a','b','c','d','e','f','g','h','i','j') DEFAULT NULL );")
+	tk.MustExec("insert into t values('i'), ('j');")
+	tk.MustExec("CREATE TABLE `s` (   `a` char(1) DEFAULT NULL,   KEY `a` (`a`) )")
+	tk.MustExec("insert into s values('i'), ('j');")
+	tk.MustQuery("select /*+ inl_hash_join(s)*/ t.a from t left join s on t.a = s.a;").Check(testkit.Rows("i", "j"))
+	tk.MustQuery("show warnings").Check(testkit.Rows())
+	tk.MustQuery("select /*+ inl_join(s)*/ t.a from t left join s on t.a = s.a;").Check(testkit.Rows("i", "j"))
+	tk.MustQuery("show warnings").Check(testkit.Rows())
+}
diff --git a/util/codec/codec_test.go b/util/codec/codec_test.go
index eb0fff931be78..8649aa3f0fa46 100644
--- a/util/codec/codec_test.go
+++ b/util/codec/codec_test.go
@@ -1034,6 +1034,7 @@ func datumsForTest(sc *stmtctx.StatementContext) ([]types.Datum, []*types.FieldT
 		{types.Duration{Duration: time.Second, Fsp: 1}, types.NewFieldType(mysql.TypeDuration)},
 		{types.Enum{Name: "a", Value: 1}, &types.FieldType{Tp: mysql.TypeEnum, Elems: []string{"a"}}},
 		{types.Set{Name: "a", Value: 1}, &types.FieldType{Tp: mysql.TypeSet, Elems: []string{"a"}}},
+		{types.Set{Name: "f", Value: 32}, &types.FieldType{Tp: mysql.TypeSet, Elems: []string{"a", "b", "c", "d", "e", "f"}}},
 		{types.BinaryLiteral{100}, &types.FieldType{Tp: mysql.TypeBit, Flen: 8}},
 		{json.CreateBinary("abc"), types.NewFieldType(mysql.TypeJSON)},
 		{int64(1), types.NewFieldType(mysql.TypeYear)},