Repository: apache/shardingsphere (2026-04-15T05:45:42Z)

Issue:
pipeline migration:  parse binlog error when JSON column in increment task 
### Which version of ShardingSphere did you use?

5.5.3-SNAPSHOT

### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?

 ShardingSphere-Proxy

### Expected behavior

Binlog parser should correctly handle JSON columns containing the literal value 'null'

### Actual behavior

**Binlog parser throws UnsupportedSQLOperationException when encountering a MySQL JSON column containing the literal string value 'null'**

Error occurs with this sequence:


CREATE TABLE  reproduce_json_parse_error (
   id bigint NOT NULL AUTO_INCREMENT,
  settlement_assets json NOT NULL,
  PRIMARY KEY (id)
);

INSERT INTO reproduce_json_parse_error(settlement_assets) VALUES('null');

**Error message:**
org.apache.shardingsphere.infra.exception.generic.UnsupportedSQLOperationException: Unsupported SQL operation: 4

### More 

- When the parsing error occurs, the binlog client becomes corrupted

- Current implementation attempts to skip the error event but fails to properly skip unresolved bytes

- This causes incorrect parsing of subsequent events






Discussion:
After further verification, this is confirmed to be a real bug rather than a usage issue.

Validation:

1. In the E2E scenario, I simulated inserting a top-level JSON literal `null` into a MySQL JSON column, and the exception can be reproduced consistently. The key stack trace is:

```text
Caused by: org.apache.shardingsphere.infra.exception.generic.UnsupportedSQLOperationException: Unsupported SQL operation: 4.

at org.apache.shardingsphere.database.protocol.mysql.packet.binlog.row.column.value.string.MySQLJsonValueDecoder.decodeValue(MySQLJsonValueDecoder.java:93)
at org.apache.shardingsphere.database.protocol.mysql.packet.binlog.row.column.value.string.MySQLJsonValueDecoder.decode(MySQLJsonValueDecoder.java:47)
at org.apache.shardingsphere.database.protocol.mysql.packet.binlog.row.column.value.string.MySQLJsonBinlogProtocolValue.read(MySQLJsonBinlogProtocolValue.java:45)
at org.apache.shardingsphere.database.protocol.mysql.packet.binlog.row.MySQLBinlogRowsEventPacket.readRow(MySQLBinlogRowsEventPacket.java:107)
at org.apache.shardingsphere.database.protocol.mysql.packet.binlog.row.MySQLBinlogRowsEventPacket.readRows(MySQLBinlogRowsEventPacket.java:95)
at org.apache.shardingsphere.data.pipeline.mysql.ingest.incremental.client.netty.MySQLBinlogEventPacketDecoder.decodeUpdateRowsEventV2(MySQLBinlogEventPacketDecoder.java:209)
at org.apache.shardingsphere.data.pipeline.mysql.ingest.incremental.client.netty.MySQLBinlogEventPacketDecoder.decodeEvent(MySQLBinlogEventPacketDecoder.java:164)
at org.apache.shardingsphere.data.pipeline.mysql.ingest.incremental.client.netty.MySQLBinlogEventPacketDecoder.decode(MySQLBinlogEventPacketDecoder.java:77)
```

Conclusion and root cause:

- Incremental MySQL migration relies on binlog parsing, and the failure happens in the MySQL binary JSON decoder.
- The current implementation already supports JSON literals `null/true/false` when they appear inside objects or arrays, but it misses the branch for top-level literals.
- In MySQL binary JSON, the literal type code is `0x04`, so a top-level `null` falls into `Unsupported SQL operation: 4`.

Fix summary:

- Add top-level JSON literal decoding support in `MySQLJsonValueDecoder`.
- Add unit tests for top-level `null / true / false`, and also cover other top-level cases such as `"hello"`, `123`, and `45.6`.
- Adjust the pipeline E2E data so that only one inserted row and one update use the JSON literal `null`, while all other rows continue to use normal JSON values.

This should fix the issue reported here, where a MySQL JSON literal `null` causes incremental binlog parsing to fail.




Patch:
diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md
index a21d81de55720..8e458f6dc36fe 100644
--- a/RELEASE-NOTES.md
+++ b/RELEASE-NOTES.md
@@ -17,6 +17,7 @@
 1. Proxy: Resolve openGauss batch bind parameter types before reading values - [#38390](https://github.com/apache/shardingsphere/pull/38390)
 1. Proxy: Fix primary key metadata loss for aliased columns in mysql prepare probe path - [#38517](https://github.com/apache/shardingsphere/pull/38517)
 1. Sharding: Fix incorrect routing when irrelevant sharding conditions are present - [#38527](https://github.com/apache/shardingsphere/pull/38527)
+1. Pipeline: Fix MySQL JSON literal decoding in migration - [#38622](https://github.com/apache/shardingsphere/pull/38622)
 
 ### Enhancements
 
diff --git a/database/protocol/dialect/mysql/src/main/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoder.java b/database/protocol/dialect/mysql/src/main/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoder.java
index 74ae5ce340a9c..4e8201e16dfdc 100644
--- a/database/protocol/dialect/mysql/src/main/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoder.java
+++ b/database/protocol/dialect/mysql/src/main/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoder.java
@@ -89,6 +89,9 @@ private static void decodeValue(final int type, final int offset, final ByteBuf
                 case JsonValueTypes.STRING:
                     outputString(decodeString(byteBuf.slice()), stringBuilder);
                     break;
+                case JsonValueTypes.LITERAL:
+                    outputLiteral(byteBuf.readUnsignedByte(), stringBuilder);
+                    break;
                 default:
                     throw new UnsupportedSQLOperationException(String.valueOf(type));
             }


Test patch:
diff --git a/database/protocol/dialect/mysql/src/test/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoderTest.java b/database/protocol/dialect/mysql/src/test/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoderTest.java
index c28fd84872287..c4d48d1b02ad8 100644
--- a/database/protocol/dialect/mysql/src/test/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoderTest.java
+++ b/database/protocol/dialect/mysql/src/test/java/org/apache/shardingsphere/database/protocol/mysql/packet/binlog/row/column/value/string/MySQLJsonValueDecoderTest.java
@@ -275,9 +275,15 @@ private byte[] codecDataLength(final int length) {
     
     private static Stream<Arguments> decodeTopLevelScalarArguments() {
         return Stream.of(
+                Arguments.of("decode top-level literal null", mockTopLevelLiteralByteBuf(JsonValueTypes.LITERAL_NULL), "null"),
+                Arguments.of("decode top-level literal true", mockTopLevelLiteralByteBuf(JsonValueTypes.LITERAL_TRUE), "true"),
+                Arguments.of("decode top-level literal false", mockTopLevelLiteralByteBuf(JsonValueTypes.LITERAL_FALSE), "false"),
+                Arguments.of("decode top-level string", mockTopLevelStringByteBuf("hello"), "\"hello\""),
+                Arguments.of("decode top-level string with escaped chars", mockTopLevelStringByteBuf("a\"\\b"), "\"a\\\"\\\\b\""),
+                Arguments.of("decode top-level int32", mockTopLevelInt32ByteBuf(123), "123"),
+                Arguments.of("decode top-level double", mockTopLevelDoubleByteBuf(45.6D), "45.6"),
                 Arguments.of("decode int16 top-level", mockTopLevelInt16ByteBuf(), "-32768"),
-                Arguments.of("decode uint16 top-level", mockTopLevelUInt16ByteBuf(), "65535"),
-                Arguments.of("decode string with escaped chars", mockTopLevelStringByteBuf(), "\"a\\\"\\\\b\""));
+                Arguments.of("decode uint16 top-level", mockTopLevelUInt16ByteBuf(), "65535"));
     }
     
     private static Stream<Arguments> decodeUnsupportedArguments() {
@@ -334,6 +340,20 @@ private static ByteBuf mockTopLevelInt16ByteBuf() {
         return result;
     }
     
+    private static ByteBuf mockTopLevelLiteralByteBuf(final byte value) {
+        ByteBuf result = Unpooled.buffer();
+        result.writeByte(JsonValueTypes.LITERAL);
+        result.writeByte(value);
+        return result;
+    }
+    
+    private static ByteBuf mockTopLevelInt32ByteBuf(final int value) {
+        ByteBuf result = Unpooled.buffer();
+        result.writeByte(JsonValueTypes.INT32);
+        result.writeIntLE(value);
+        return result;
+    }
+    
     private static ByteBuf mockTopLevelUInt16ByteBuf() {
         ByteBuf result = Unpooled.buffer();
         result.writeByte(JsonValueTypes.UINT16);
@@ -341,11 +361,18 @@ private static ByteBuf mockTopLevelUInt16ByteBuf() {
         return result;
     }
     
-    private static ByteBuf mockTopLevelStringByteBuf() {
+    private static ByteBuf mockTopLevelDoubleByteBuf(final double value) {
+        ByteBuf result = Unpooled.buffer();
+        result.writeByte(JsonValueTypes.DOUBLE);
+        result.writeDoubleLE(value);
+        return result;
+    }
+    
+    private static ByteBuf mockTopLevelStringByteBuf(final String value) {
         ByteBuf result = Unpooled.buffer();
         result.writeByte(JsonValueTypes.STRING);
-        result.writeByte("a\"\\b".length());
-        result.writeBytes("a\"\\b".getBytes());
+        result.writeByte(value.length());
+        result.writeBytes(value.getBytes());
         return result;
     }
     
diff --git a/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/cases/task/E2EIncrementalTask.java b/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/cases/task/E2EIncrementalTask.java
index c837f0717e560..14f8b587e4970 100644
--- a/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/cases/task/E2EIncrementalTask.java
+++ b/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/cases/task/E2EIncrementalTask.java
@@ -47,6 +47,7 @@
 import java.util.LinkedList;
 import java.util.List;
 import java.util.UUID;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.ThreadLocalRandom;
 
 @RequiredArgsConstructor
@@ -75,6 +76,8 @@ public final class E2EIncrementalTask implements Runnable {
     
     private final int loopCount;
     
+    private final AtomicBoolean generatedJsonLiteralNullForUpdate = new AtomicBoolean(false);
+    
     @Override
     public void run() {
         try {
@@ -152,7 +155,7 @@ private void updateOrderById(final Object orderId) throws SQLException {
             Object[] parameters = {"中文测试", randomInt, randomInt, randomInt, randomUnsignedInt, randomUnsignedInt, randomUnsignedInt,
                     randomUnsignedInt, 1.0F, 1.0, new BigDecimal("999"), now, now, now.toLocalDate(), now.toLocalTime(), Year.now().getValue() + 1,
                     new byte[]{-1, 0, 1}, new byte[]{1, 2, -1, -3},
-                    "D".getBytes(), "A".getBytes(), "T".getBytes(), "E", "text", "mediumText", "3", "3", PipelineCaseHelper.generateJsonString(32, true), orderId};
+                    "D".getBytes(), "A".getBytes(), "T".getBytes(), "E", "text", "mediumText", "3", "3", generateMySQLJsonValue(), orderId};
             log.info("update sql: {}, params: {}", sql, parameters);
             DataSourceExecuteUtils.execute(dataSource, sql, parameters);
             return;
@@ -180,6 +183,10 @@ private void updateOrderById(final Object orderId) throws SQLException {
         }
     }
     
+    private String generateMySQLJsonValue() {
+        return generatedJsonLiteralNullForUpdate.compareAndSet(false, true) ? PipelineCaseHelper.generateJsonLiteralNull() : PipelineCaseHelper.generateJsonString(32, true);
+    }
+    
     private List<String> ignoreShardingColumns(final List<String> columnNames) {
         return new ArrayList<>(columnNames.subList(2, columnNames.size()));
     }
diff --git a/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/dao/order/large/sqlbuilder/MySQLIntPkLargeOrderSQLBuilder.java b/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/dao/order/large/sqlbuilder/MySQLIntPkLargeOrderSQLBuilder.java
index 3273bccad36af..8a5235c8a0582 100644
--- a/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/dao/order/large/sqlbuilder/MySQLIntPkLargeOrderSQLBuilder.java
+++ b/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/dao/order/large/sqlbuilder/MySQLIntPkLargeOrderSQLBuilder.java
@@ -103,7 +103,7 @@ public List<Object[]> generateInsertData(final KeyGenerateAlgorithm keyGenerateA
                     "1", "t", "e", "s", "t",
                     PipelineCaseHelper.generateString(2), "☠️x☺️x✋x☹️", PipelineCaseHelper.generateString(1),
                     "1", "2",
-                    PipelineCaseHelper.generateJsonString(32, false)
+                    0 == i ? PipelineCaseHelper.generateJsonLiteralNull() : PipelineCaseHelper.generateJsonString(32, false)
             };
             result.add(params);
         }
diff --git a/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/framework/helper/PipelineCaseHelper.java b/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/framework/helper/PipelineCaseHelper.java
index f71e14865362d..2ff2fa76e28e9 100644
--- a/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/framework/helper/PipelineCaseHelper.java
+++ b/test/e2e/operation/pipeline/src/test/java/org/apache/shardingsphere/test/e2e/operation/pipeline/framework/helper/PipelineCaseHelper.java
@@ -75,6 +75,15 @@ public static String generateJsonString(final int length, final boolean useUnico
         return String.format("{\"test\":\"%s\"}", value);
     }
     
+    /**
+     * Generate json literal null.
+     *
+     * @return json literal null
+     */
+    public static String generateJsonLiteralNull() {
+        return "null";
+    }
+    
     /**
      * Generate float value.
      *