Repository: go-kratos/kratos
Pull request #1989 (2022-05-17 06:19:16)

fix: query params: parse list failed, unsupported message type: "google.protobuf.Struct" #1967
<!--
🎉 Thanks for sending a pull request to Kratos! Here are some tips for you:

1. If this is your first time contributing to Kratos, please read our contribution guide: https://go-kratos.dev/en/docs/community/contribution/
2. Ensure you have added or ran the appropriate tests and lint for your PR, please use `make lint` and `make test` before filing your PR, use `make clean` to tidy your go mod.
3. If the PR is unfinished, you may need mark it as a WIP(Work In Progress) PR or draft PR
4. Please use a conventional commits format title: `<type>[optional scope]: <description>`
    
    Some suggestion on <type>:

    fix: A bug fix
    feat: A new feature
    test: Adding missing tests or correcting existing tests
    refactor: A code change that neither fixes a bug nor adds a feature
    break: Changes has break change

    docs: Documentation only changes
    deps: Changes external dependencies
    style: Changes that do not affect the meaning of the code (white-space, formatting, etc)
    chore Daily work, examples, etc.
    ci: Changes to our CI configuration files and scripts
-->

#### Description (what this PR does / why we need it):
<!--
* The description should include the motivation for this PR or contrast this with previous behavior
-->
fix: query params: parse list failed, unsupported message type: \"google.protobuf.Struct\" #1967

#### Which issue(s) this PR fixes (resolves / be part of):
<!--
* Automatically closes linked issue when PR is merged.
* If you PR is not fully resolved issue, please use `part of #<issue number>` instead.

Usage: `fixes/resolves #<issue number>`, or `fixes/resolves (paste link of issue)`.
-->
fixes #

#1967

#### Other special notes for reviewer:
<!--
* Somethings that need extra attention for reviewer
* Some additional notes, TODO list, etc.
-->


shenqidebaozi: Please complete the title information
codecov-commenter: # [Codecov](https://codecov.io/gh/go-kratos/kratos/pull/1989?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos) Report
> Merging [#1989](https://codecov.io/gh/go-kratos/kratos/pull/1989?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos) (0ced04c) into [main](https://codecov.io/gh/go-kratos/kratos/commit/ed144f6813c179b7eb0566d5b9ccb6a1377e4a1d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos) (ed144f6) will **increase** coverage by `0.02%`.
> The diff coverage is `60.00%`.

> :exclamation: Current head 0ced04c differs from pull request most recent head 5c70292. Consider uploading reports for the commit 5c70292 to get more accurate results

```diff
@@            Coverage Diff             @@
##             main    #1989      +/-   ##
==========================================
+ Coverage   80.71%   80.74%   +0.02%     
==========================================
  Files          82       82              
  Lines        3635     3640       +5     
==========================================
+ Hits         2934     2939       +5     
+ Misses        494      493       -1     
- Partials      207      208       +1     
```


| [Impacted Files](https://codecov.io/gh/go-kratos/kratos/pull/1989?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos) | Coverage Δ | |
|---|---|---|
| [encoding/form/proto\_decode.go](https://codecov.io/gh/go-kratos/kratos/pull/1989/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos#diff-ZW5jb2RpbmcvZm9ybS9wcm90b19kZWNvZGUuZ28=) | `54.01% <60.00%> (+0.13%)` | :arrow_up: |
| [internal/context/context.go](https://codecov.io/gh/go-kratos/kratos/pull/1989/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos#diff-aW50ZXJuYWwvY29udGV4dC9jb250ZXh0Lmdv) | `100.00% <0.00%> (+2.98%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/go-kratos/kratos/pull/1989?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/go-kratos/kratos/pull/1989?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos). Last update [ed144f6...5c70292](https://codecov.io/gh/go-kratos/kratos/pull/1989?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-kratos).


Patch:
diff --git a/encoding/form/proto_decode.go b/encoding/form/proto_decode.go
index df2163bc..8b4b3159 100644
--- a/encoding/form/proto_decode.go
+++ b/encoding/form/proto_decode.go
@@ -9,6 +9,7 @@ import (
 	"strings"
 	"time"
 
+	"google.golang.org/protobuf/encoding/protojson"
 	"google.golang.org/protobuf/types/known/structpb"
 
 	"google.golang.org/genproto/protobuf/field_mask"
@@ -297,6 +298,12 @@ func parseMessage(md protoreflect.MessageDescriptor, value string) (protoreflect
 			return protoreflect.Value{}, err
 		}
 		msg = fm
+	case "google.protobuf.Struct":
+		var v structpb.Struct
+		if err := protojson.Unmarshal([]byte(value), &v); err != nil {
+			return protoreflect.Value{}, err
+		}
+		msg = &v
 	default:
 		return protoreflect.Value{}, fmt.Errorf("unsupported message type: %q", string(md.FullName()))
 	}


Test patch:
diff --git a/encoding/form/form_test.go b/encoding/form/form_test.go
index a1840b7f..f437ddf1 100644
--- a/encoding/form/form_test.go
+++ b/encoding/form/form_test.go
@@ -11,6 +11,7 @@ import (
 
 	"github.com/go-kratos/kratos/v2/encoding"
 	"github.com/go-kratos/kratos/v2/internal/testdata/complex"
+	ectest "github.com/go-kratos/kratos/v2/internal/testdata/encoding"
 )
 
 type LoginRequest struct {
@@ -156,3 +157,24 @@ func TestProtoEncodeDecode(t *testing.T) {
 		t.Errorf("expect %v, got %v", "5566", in2.Simples[1])
 	}
 }
+
+func TestDecodeStructPb(t *testing.T) {
+	req := new(ectest.StructPb)
+	query := `data={"name":"kratos"}&data_list={"name1": "kratos"}&data_list={"name2": "go-kratos"}`
+	if err := encoding.GetCodec(contentType).Unmarshal([]byte(query), req); err != nil {
+		t.Errorf("expect %v, got %v", nil, err)
+	}
+	if !reflect.DeepEqual("kratos", req.Data.GetFields()["name"].GetStringValue()) {
+		t.Errorf("except %v, got %v", "kratos", req.Data.GetFields()["name"].GetStringValue())
+	}
+	if len(req.DataList) != 2 {
+		t.Errorf("execpt %v, got %v", 2, len(req.DataList))
+		return
+	}
+	if !reflect.DeepEqual("kratos", req.DataList[0].GetFields()["name1"].GetStringValue()) {
+		t.Errorf("except %v, got %v", "kratos", req.Data.GetFields()["name1"].GetStringValue())
+	}
+	if !reflect.DeepEqual("go-kratos", req.DataList[1].GetFields()["name2"].GetStringValue()) {
+		t.Errorf("except %v, got %v", "go-kratos", req.Data.GetFields()["name2"].GetStringValue())
+	}
+}
diff --git a/internal/testdata/encoding/test.pb.go b/internal/testdata/encoding/test.pb.go
index cf3d97cc..b5b6c5d4 100644
--- a/internal/testdata/encoding/test.pb.go
+++ b/internal/testdata/encoding/test.pb.go
@@ -1,7 +1,7 @@
 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
 // 	protoc-gen-go v1.27.1
-// 	protoc        v3.17.3
+// 	protoc        v3.19.4
 // source: encoding/test.proto
 
 package encoding
@@ -9,6 +9,7 @@ package encoding
 import (
 	protoreflect "google.golang.org/protobuf/reflect/protoreflect"
 	protoimpl "google.golang.org/protobuf/runtime/protoimpl"
+	structpb "google.golang.org/protobuf/types/known/structpb"
 	reflect "reflect"
 	sync "sync"
 )
@@ -91,24 +92,88 @@ func (x *TestModel) GetAttrs() map[string]string {
 	return nil
 }
 
+type StructPb struct {
+	state         protoimpl.MessageState
+	sizeCache     protoimpl.SizeCache
+	unknownFields protoimpl.UnknownFields
+
+	Data     *structpb.Struct   `protobuf:"bytes,1,opt,name=data,proto3" json:"data,omitempty"`
+	DataList []*structpb.Struct `protobuf:"bytes,2,rep,name=data_list,json=dataList,proto3" json:"data_list,omitempty"`
+}
+
+func (x *StructPb) Reset() {
+	*x = StructPb{}
+	if protoimpl.UnsafeEnabled {
+		mi := &file_encoding_test_proto_msgTypes[1]
+		ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+		ms.StoreMessageInfo(mi)
+	}
+}
+
+func (x *StructPb) String() string {
+	return protoimpl.X.MessageStringOf(x)
+}
+
+func (*StructPb) ProtoMessage() {}
+
+func (x *StructPb) ProtoReflect() protoreflect.Message {
+	mi := &file_encoding_test_proto_msgTypes[1]
+	if protoimpl.UnsafeEnabled && x != nil {
+		ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+		if ms.LoadMessageInfo() == nil {
+			ms.StoreMessageInfo(mi)
+		}
+		return ms
+	}
+	return mi.MessageOf(x)
+}
+
+// Deprecated: Use StructPb.ProtoReflect.Descriptor instead.
+func (*StructPb) Descriptor() ([]byte, []int) {
+	return file_encoding_test_proto_rawDescGZIP(), []int{1}
+}
+
+func (x *StructPb) GetData() *structpb.Struct {
+	if x != nil {
+		return x.Data
+	}
+	return nil
+}
+
+func (x *StructPb) GetDataList() []*structpb.Struct {
+	if x != nil {
+		return x.DataList
+	}
+	return nil
+}
+
 var File_encoding_test_proto protoreflect.FileDescriptor
 
 var file_encoding_test_proto_rawDesc = []byte{
 	0x0a, 0x13, 0x65, 0x6e, 0x63, 0x6f, 0x64, 0x69, 0x6e, 0x67, 0x2f, 0x74, 0x65, 0x73, 0x74, 0x2e,
-	0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x04, 0x74, 0x65, 0x73, 0x74, 0x22, 0xb3, 0x01, 0x0a, 0x0a,
-	0x74, 0x65, 0x73, 0x74, 0x5f, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64,
-	0x18, 0x01, 0x20, 0x01, 0x28, 0x03, 0x52, 0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61,
-	0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x14,
-	0x0a, 0x05, 0x68, 0x6f, 0x62, 0x62, 0x79, 0x18, 0x03, 0x20, 0x03, 0x28, 0x09, 0x52, 0x05, 0x68,
-	0x6f, 0x62, 0x62, 0x79, 0x12, 0x31, 0x0a, 0x05, 0x61, 0x74, 0x74, 0x72, 0x73, 0x18, 0x04, 0x20,
-	0x03, 0x28, 0x0b, 0x32, 0x1b, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x5f,
-	0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x2e, 0x41, 0x74, 0x74, 0x72, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79,
-	0x52, 0x05, 0x61, 0x74, 0x74, 0x72, 0x73, 0x1a, 0x38, 0x0a, 0x0a, 0x41, 0x74, 0x74, 0x72, 0x73,
-	0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01,
-	0x28, 0x09, 0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65,
-	0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38,
-	0x01, 0x42, 0x0d, 0x5a, 0x0b, 0x2e, 0x2e, 0x2f, 0x65, 0x6e, 0x63, 0x6f, 0x64, 0x69, 0x6e, 0x67,
-	0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
+	0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x04, 0x74, 0x65, 0x73, 0x74, 0x1a, 0x1c, 0x67, 0x6f, 0x6f,
+	0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x73, 0x74, 0x72,
+	0x75, 0x63, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0xb3, 0x01, 0x0a, 0x0a, 0x74, 0x65,
+	0x73, 0x74, 0x5f, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01,
+	0x20, 0x01, 0x28, 0x03, 0x52, 0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65,
+	0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x14, 0x0a, 0x05,
+	0x68, 0x6f, 0x62, 0x62, 0x79, 0x18, 0x03, 0x20, 0x03, 0x28, 0x09, 0x52, 0x05, 0x68, 0x6f, 0x62,
+	0x62, 0x79, 0x12, 0x31, 0x0a, 0x05, 0x61, 0x74, 0x74, 0x72, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28,
+	0x0b, 0x32, 0x1b, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x5f, 0x6d, 0x6f,
+	0x64, 0x65, 0x6c, 0x2e, 0x41, 0x74, 0x74, 0x72, 0x73, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x05,
+	0x61, 0x74, 0x74, 0x72, 0x73, 0x1a, 0x38, 0x0a, 0x0a, 0x41, 0x74, 0x74, 0x72, 0x73, 0x45, 0x6e,
+	0x74, 0x72, 0x79, 0x12, 0x10, 0x0a, 0x03, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,
+	0x52, 0x03, 0x6b, 0x65, 0x79, 0x12, 0x14, 0x0a, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x18, 0x02,
+	0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x3a, 0x02, 0x38, 0x01, 0x22,
+	0x6d, 0x0a, 0x08, 0x53, 0x74, 0x72, 0x75, 0x63, 0x74, 0x50, 0x62, 0x12, 0x2b, 0x0a, 0x04, 0x64,
+	0x61, 0x74, 0x61, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x67, 0x6f, 0x6f, 0x67,
+	0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x53, 0x74, 0x72, 0x75,
+	0x63, 0x74, 0x52, 0x04, 0x64, 0x61, 0x74, 0x61, 0x12, 0x34, 0x0a, 0x09, 0x64, 0x61, 0x74, 0x61,
+	0x5f, 0x6c, 0x69, 0x73, 0x74, 0x18, 0x02, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x67, 0x6f,
+	0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x53, 0x74,
+	0x72, 0x75, 0x63, 0x74, 0x52, 0x08, 0x64, 0x61, 0x74, 0x61, 0x4c, 0x69, 0x73, 0x74, 0x42, 0x0d,
+	0x5a, 0x0b, 0x2e, 0x2e, 0x2f, 0x65, 0x6e, 0x63, 0x6f, 0x64, 0x69, 0x6e, 0x67, 0x62, 0x06, 0x70,
+	0x72, 0x6f, 0x74, 0x6f, 0x33,
 }
 
 var (
@@ -123,18 +188,22 @@ func file_encoding_test_proto_rawDescGZIP() []byte {
 	return file_encoding_test_proto_rawDescData
 }
 
-var file_encoding_test_proto_msgTypes = make([]protoimpl.MessageInfo, 2)
+var file_encoding_test_proto_msgTypes = make([]protoimpl.MessageInfo, 3)
 var file_encoding_test_proto_goTypes = []interface{}{
-	(*TestModel)(nil), // 0: test.test_model
-	nil,               // 1: test.test_model.AttrsEntry
+	(*TestModel)(nil),       // 0: test.test_model
+	(*StructPb)(nil),        // 1: test.StructPb
+	nil,                     // 2: test.test_model.AttrsEntry
+	(*structpb.Struct)(nil), // 3: google.protobuf.Struct
 }
 var file_encoding_test_proto_depIdxs = []int32{
-	1, // 0: test.test_model.attrs:type_name -> test.test_model.AttrsEntry
-	1, // [1:1] is the sub-list for method output_type
-	1, // [1:1] is the sub-list for method input_type
-	1, // [1:1] is the sub-list for extension type_name
-	1, // [1:1] is the sub-list for extension extendee
-	0, // [0:1] is the sub-list for field type_name
+	2, // 0: test.test_model.attrs:type_name -> test.test_model.AttrsEntry
+	3, // 1: test.StructPb.data:type_name -> google.protobuf.Struct
+	3, // 2: test.StructPb.data_list:type_name -> google.protobuf.Struct
+	3, // [3:3] is the sub-list for method output_type
+	3, // [3:3] is the sub-list for method input_type
+	3, // [3:3] is the sub-list for extension type_name
+	3, // [3:3] is the sub-list for extension extendee
+	0, // [0:3] is the sub-list for field type_name
 }
 
 func init() { file_encoding_test_proto_init() }
@@ -155,6 +224,18 @@ func file_encoding_test_proto_init() {
 				return nil
 			}
 		}
+		file_encoding_test_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {
+			switch v := v.(*StructPb); i {
+			case 0:
+				return &v.state
+			case 1:
+				return &v.sizeCache
+			case 2:
+				return &v.unknownFields
+			default:
+				return nil
+			}
+		}
 	}
 	type x struct{}
 	out := protoimpl.TypeBuilder{
@@ -162,7 +243,7 @@ func file_encoding_test_proto_init() {
 			GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
 			RawDescriptor: file_encoding_test_proto_rawDesc,
 			NumEnums:      0,
-			NumMessages:   2,
+			NumMessages:   3,
 			NumExtensions: 0,
 			NumServices:   0,
 		},
diff --git a/internal/testdata/encoding/test.proto b/internal/testdata/encoding/test.proto
index de02c783..ad39a7f4 100644
--- a/internal/testdata/encoding/test.proto
+++ b/internal/testdata/encoding/test.proto
@@ -4,9 +4,16 @@ package test;
 
 option go_package = "../encoding";
 
+import "google/protobuf/struct.proto";
+
 message test_model {
   int64 id = 1;
   string name = 2;
   repeated string hobby = 3;
   map<string, string> attrs = 4;
-}
\ No newline at end of file
+}
+
+message StructPb {
+  google.protobuf.Struct data = 1;
+  repeated  google.protobuf.Struct data_list = 2;
+}