Repository: pingcap/tidb (Go)

Issue:
Partition pruning is not as expected when condition exceeds the range of column type
## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)
```sql
DROP TABLE IF EXISTS t4;
CREATE TABLE `t4` (
  a bit(1) DEFAULT NULL,
  b int(11) DEFAULT NULL
) PARTITION BY HASH(a)
PARTITIONS 3;
EXPLAIN SELECT * FROM t4 WHERE a = 5;
```
<!-- a step by step guide for reproducing the bug. -->

### 2. What did you expect to see? (Required)
in mysql 8.0，the partition in the plan is null
```
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | t4    | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    1 |   100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
```
### 3. What did you see instead (Required)
```
+-------------------------+----------+-----------+------------------------+--------------------------------+
| id                      | estRows  | task      | access object          | operator info                  |
+-------------------------+----------+-----------+------------------------+--------------------------------+
| Selection_6             | 8000.00  | root      |                        | eq(test.t4.a, 5)               |
| └─TableReader_8         | 10000.00 | root      |                        | data:TableFullScan_7           |
|   └─TableFullScan_7     | 10000.00 | cop[tikv] | table:t4, partition:p2 | keep order:false, stats:pseudo |
+-------------------------+----------+-----------+------------------------+--------------------------------+
```
### 4. What is your TiDB version? (Required)
master
```
Release Version: v4.0.0-beta.2-855-g4db32ba
Edition: Community
Git Commit Hash: 4db32ba6fe6400d2e6db9cc225c22e6fae2949c9
Git Branch: HEAD
UTC Build Time: 2020-07-29 05:16:59
GoVersion: go1.13.7
Race Enabled: false
TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
Check Table Before Drop: false
```
<!-- Paste the output of SELECT tidb_version() -->




Discussion:
This should be an 'impossible where'. MySQL can actually detect it in other cases, but seemingly not this one.

There is a similar issue in https://github.com/pingcap/tidb/issues/17786
/assign
Because of the [PR#20241](https://github.com/pingcap/tidb/pull/20241), we can see
```
mysql> EXPLAIN SELECT * FROM t4 WHERE a = 5;
ERROR 1406 (22001): Data Too Long, field len 1
```
## Please edit this comment or add a new comment to complete the following information

### Not a bug

1. Remove the 'type/bug' label
2. Add notes to indicate why it is not a bug

### Duplicate bug

1. Add the 'type/duplicate' label
2. Add the link to the original bug

### Bug

Note: Make Sure that 'component', and 'severity' labels are added
Example for how to fill out the template: https://github.com/pingcap/tidb/issues/20100

#### 1. Root Cause Analysis (RCA) (optional) 
<!-- Write down the reason why this bug occurs -->

#### 2. Symptom (optional)

<!-- What will the user see when this bug occurs. The error message may be in the terminal, log or monitoring -->

#### 3. All Trigger Conditions (optional)

<!-- All the user scenarios that may trigger this bug -->

#### 4. Workaround (optional)

#### 5. Affected versions

<!--
In the format of [start_version:end_version], multiple version ranges are
accepted. If the bug only affects the unreleased version, please input:
"unreleased". For example:

Notes:
  1. Do not use any white spaces in '[]'.
  2. The range in '[]' is a closed interval
  3. The version format is `v$Major.$Minor.$Patch`, the $Majoy and $Minor
     number in a version range should be the same. [v3.0.1:v3.1.2] is
     invalid because the $Minor number of the version range is different.

Example 1: [v3.0.1:v3.0.5], [v4.0.1:v4.0.5]
Example 2: unreleased
-->

#### 6. Fixed versions

<!--
The first released version that contains this fix in each minor version. If the bug's affected version has been released, the fixed version should be a detailed version number; If the bug doesn't affect any released version, the fixed version can be "master";  

Example 1: v3.0.13, v4.0.5
Example 2: master
-->



Patch:
diff --git a/util/ranger/ranger.go b/util/ranger/ranger.go
index 8edc9c775f438..3a1ab70c5ab0b 100644
--- a/util/ranger/ranger.go
+++ b/util/ranger/ranger.go
@@ -93,8 +93,11 @@ func convertPoint(sc *stmtctx.StatementContext, point point, tp *types.FieldType
 	}
 	casted, err := point.value.ConvertTo(sc, tp)
 	if err != nil {
-		// see issue #20101: overflow when converting integer to year
-		if tp.Tp != mysql.TypeYear || !terror.ErrorEqual(err, types.ErrInvalidYear) {
+		if tp.Tp == mysql.TypeYear && terror.ErrorEqual(err, types.ErrInvalidYear) {
+			// see issue #20101: overflow when converting integer to year
+		} else if tp.Tp == mysql.TypeBit && terror.ErrorEqual(err, types.ErrDataTooLong) {
+			// see issue #19067: we should ignore the types.ErrDataTooLong when we convert value to TypeBit value
+		} else {
 			return point, errors.Trace(err)
 		}
 	}


Test patch:
diff --git a/util/ranger/ranger_test.go b/util/ranger/ranger_test.go
index 9f00d6ddaa749..1799862f9c8d9 100644
--- a/util/ranger/ranger_test.go
+++ b/util/ranger/ranger_test.go
@@ -1366,6 +1366,43 @@ func (s *testRangerSuite) TestPrefixIndexMultiColDNF(c *C) {
 	}
 }
 
+func (s *testRangerSuite) TestIndexRangeForBit(c *C) {
+	defer testleak.AfterTest(c)()
+	dom, store, err := newDomainStoreWithBootstrap(c)
+	defer func() {
+		dom.Close()
+		store.Close()
+	}()
+	c.Assert(err, IsNil)
+	testKit := testkit.NewTestKit(c, store)
+	testKit.MustExec("use test;")
+	testKit.MustExec("drop table if exists t;")
+	testKit.MustExec("CREATE TABLE `t` (" +
+		"a bit(1) DEFAULT NULL," +
+		"b int(11) DEFAULT NULL" +
+		") PARTITION BY HASH(a)" +
+		"PARTITIONS 3;")
+	testKit.MustExec("insert ignore into t values(-1, -1), (0, 0), (1, 1), (3, 3);")
+	testKit.MustExec("analyze table t;")
+
+	var input []string
+	var output []struct {
+		SQL    string
+		Plan   []string
+		Result []string
+	}
+	s.testData.GetTestCases(c, &input, &output)
+	for i, tt := range input {
+		s.testData.OnRecord(func() {
+			output[i].SQL = tt
+			output[i].Plan = s.testData.ConvertRowsToStrings(testKit.MustQuery("explain " + tt).Rows())
+			output[i].Result = s.testData.ConvertRowsToStrings(testKit.MustQuery(tt).Rows())
+		})
+		testKit.MustQuery("explain " + tt).Check(testkit.Rows(output[i].Plan...))
+		testKit.MustQuery(tt).Check(testkit.Rows(output[i].Result...))
+	}
+}
+
 func (s *testRangerSuite) TestIndexRangeForYear(c *C) {
 	defer testleak.AfterTest(c)()
 	dom, store, err := newDomainStoreWithBootstrap(c)
diff --git a/util/ranger/testdata/ranger_suite_in.json b/util/ranger/testdata/ranger_suite_in.json
index 98c802b1921b8..0bce297979386 100644
--- a/util/ranger/testdata/ranger_suite_in.json
+++ b/util/ranger/testdata/ranger_suite_in.json
@@ -67,5 +67,22 @@
       "select * from t2 where t='aaaa';",
       "select * from t2 where t='aaaa' or t = 'a';"
     ]
+  },
+  {
+    "name": "TestIndexRangeForBit",
+    "cases": [
+      "select * from t;",
+      "select * from t where a = 0;",
+      "select * from t where a = 0 or a = 4;",
+      "select * from t where a = 1;",
+      "select * from t where a = -1;",
+      "select * from t where a = 3;",
+      "select * from t where a < 1;",
+      "select * from t where a < 3;",
+      "select * from t where a < -1;",
+      "select * from t where a > 0;",
+      "select * from t where a > -1;",
+      "select * from t where a > 3;"
+    ]
   }
 ]
diff --git a/util/ranger/testdata/ranger_suite_out.json b/util/ranger/testdata/ranger_suite_out.json
index 1cdfbbc9e276e..2a448461ce10d 100644
--- a/util/ranger/testdata/ranger_suite_out.json
+++ b/util/ranger/testdata/ranger_suite_out.json
@@ -385,5 +385,146 @@
         ]
       }
     ]
+  },
+  {
+    "Name": "TestIndexRangeForBit",
+    "Cases": [
+      {
+        "SQL": "select * from t;",
+        "Plan": [
+          "TableReader_5 4.00 root partition:all data:TableFullScan_4",
+          "└─TableFullScan_4 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0000 0",
+          "\u0001 -1",
+          "\u0001 1",
+          "\u0001 3"
+        ]
+      },
+      {
+        "SQL": "select * from t where a = 0;",
+        "Plan": [
+          "Selection_5 3.20 root  eq(test.t.a, 0)",
+          "└─TableReader_7 4.00 root partition:p0 data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0000 0"
+        ]
+      },
+      {
+        "SQL": "select * from t where a = 0 or a = 4;",
+        "Plan": [
+          "Selection_5 3.20 root  or(eq(test.t.a, 0), eq(test.t.a, 4))",
+          "└─TableReader_7 4.00 root partition:p0 data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0000 0"
+        ]
+      },
+      {
+        "SQL": "select * from t where a = 1;",
+        "Plan": [
+          "Selection_5 3.20 root  eq(test.t.a, 1)",
+          "└─TableReader_7 4.00 root partition:p1 data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0001 -1",
+          "\u0001 1",
+          "\u0001 3"
+        ]
+      },
+      {
+        "SQL": "select * from t where a = -1;",
+        "Plan": [
+          "Selection_5 3.20 root  eq(test.t.a, -1)",
+          "└─TableReader_7 4.00 root partition:dual data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": null
+      },
+      {
+        "SQL": "select * from t where a = 3;",
+        "Plan": [
+          "Selection_5 3.20 root  eq(test.t.a, 3)",
+          "└─TableReader_7 4.00 root partition:dual data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": null
+      },
+      {
+        "SQL": "select * from t where a < 1;",
+        "Plan": [
+          "Selection_5 3.20 root  lt(test.t.a, 1)",
+          "└─TableReader_7 4.00 root partition:all data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0000 0"
+        ]
+      },
+      {
+        "SQL": "select * from t where a < 3;",
+        "Plan": [
+          "Selection_5 3.20 root  lt(test.t.a, 3)",
+          "└─TableReader_7 4.00 root partition:all data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0000 0",
+          "\u0001 -1",
+          "\u0001 1",
+          "\u0001 3"
+        ]
+      },
+      {
+        "SQL": "select * from t where a < -1;",
+        "Plan": [
+          "Selection_5 3.20 root  lt(test.t.a, -1)",
+          "└─TableReader_7 4.00 root partition:dual data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": null
+      },
+      {
+        "SQL": "select * from t where a > 0;",
+        "Plan": [
+          "Selection_5 3.20 root  gt(test.t.a, 0)",
+          "└─TableReader_7 4.00 root partition:all data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0001 -1",
+          "\u0001 1",
+          "\u0001 3"
+        ]
+      },
+      {
+        "SQL": "select * from t where a > -1;",
+        "Plan": [
+          "Selection_5 3.20 root  gt(test.t.a, -1)",
+          "└─TableReader_7 4.00 root partition:all data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": [
+          "\u0000 0",
+          "\u0001 -1",
+          "\u0001 1",
+          "\u0001 3"
+        ]
+      },
+      {
+        "SQL": "select * from t where a > 3;",
+        "Plan": [
+          "Selection_5 3.20 root  gt(test.t.a, 3)",
+          "└─TableReader_7 4.00 root partition:all data:TableFullScan_6",
+          "  └─TableFullScan_6 4.00 cop[tikv] table:t keep order:false"
+        ],
+        "Result": null
+      }
+    ]
   }
 ]