Repository: cloudwego/hertz
Pull request #940 (2023-09-09 02:46:21)

feat: add disable normalizing
#### What type of PR is this?
<!--
Add one of the following kinds:

build: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
ci: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
docs: Documentation only changes
feat: A new feature
optimize: A new optimization
fix: A bug fix
perf: A code change that improves performance
refactor: A code change that neither fixes a bug nor adds a feature
style: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc)
test: Adding missing tests or correcting existing tests
chore: Changes to the build process or auxiliary tools and libraries such as documentation generation
-->

#### Check the PR title.
<!--
The description of the title will be attached in Release Notes, 
so please describe it from user-oriented, what this PR does / why we need it.
Please check your PR title with the below requirements:
-->
- [x] This PR title match the format: \<type\>(optional scope): \<description\>
- [x] The description of this PR title is user-oriented and clear enough for others to understand.
- [ ] Attach the PR updating the user documentation if the current PR requires user awareness at the usage level. [User docs repo](https://github.com/cloudwego/cloudwego.github.io)


#### (Optional) Translate the PR title into Chinese.


#### (Optional) More detailed description for this PR(en: English/zh: Chinese).
<!--
Provide more detailed info for review(e.g., it's recommended to provide perf data if this is a perf type PR).
-->
en:
zh(optional):

#### (Optional) Which issue(s) this PR fixes:
<!--
Automatically closes linked issue when PR is merged.
Eg: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
-->

#### (Optional) The PR that updates user documentation:
<!--
If the current PR requires user awareness at the usage level, please submit a PR to update user docs. [User docs repo](https://github.com/cloudwego/cloudwego.github.io)
-->

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego) Report
Patch coverage: **`62.50%`** and project coverage change: **`-0.05%`** :warning:
> Comparison is base [(`03376c9`)](https://app.codecov.io/gh/cloudwego/hertz/commit/03376c92b0cc8aaa3016aedfb8c2954fb5ac34c5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego) 81.95% compared to head [(`3d9af2c`)](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego) 81.91%.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           develop     #940      +/-   ##
===========================================
- Coverage    81.95%   81.91%   -0.05%     
===========================================
  Files           97       97              
  Lines         9625     9641      +16     
===========================================
+ Hits          7888     7897       +9     
- Misses        1263     1268       +5     
- Partials       474      476       +2     
```


| [Files Changed](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego) | Coverage Δ | |
|---|---|---|
| [pkg/protocol/http1/server.go](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego#diff-cGtnL3Byb3RvY29sL2h0dHAxL3NlcnZlci5nbw==) | `74.27% <ø> (ø)` | |
| [pkg/route/engine.go](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego#diff-cGtnL3JvdXRlL2VuZ2luZS5nbw==) | `59.79% <0.00%> (-0.76%)` | :arrow_down: |
| [pkg/app/server/option.go](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego#diff-cGtnL2FwcC9zZXJ2ZXIvb3B0aW9uLmdv) | `98.66% <100.00%> (+0.07%)` | :arrow_up: |
| [pkg/common/config/option.go](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego#diff-cGtnL2NvbW1vbi9jb25maWcvb3B0aW9uLmdv) | `100.00% <100.00%> (ø)` | |

... and [2 files with indirect coverage changes](https://app.codecov.io/gh/cloudwego/hertz/pull/940/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego)


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/cloudwego/hertz/pull/940?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cloudwego).


Patch:
diff --git a/pkg/app/server/option.go b/pkg/app/server/option.go
index bf44829..c9e3735 100644
--- a/pkg/app/server/option.go
+++ b/pkg/app/server/option.go
@@ -346,3 +346,10 @@ func WithOnConnect(fn func(ctx context.Context, conn network.Conn) context.Conte
 		o.OnConnect = fn
 	}}
 }
+
+// WithDisableHeaderNamesNormalizing is used to set whether disable header names normalizing.
+func WithDisableHeaderNamesNormalizing(disable bool) config.Option {
+	return config.Option{F: func(o *config.Options) {
+		o.DisableHeaderNamesNormalizing = disable
+	}}
+}
diff --git a/pkg/common/config/option.go b/pkg/common/config/option.go
index 03c84a0..9ef7ddf 100644
--- a/pkg/common/config/option.go
+++ b/pkg/common/config/option.go
@@ -97,6 +97,22 @@ type Options struct {
 	// The HTML template will reload according to files' changing event
 	// otherwise it will reload after AutoReloadInterval.
 	AutoReloadInterval time.Duration
+
+	// Header names are passed as-is without normalization
+	// if this option is set.
+	//
+	// Disabled header names' normalization may be useful only for proxying
+	// responses to other clients expecting case-sensitive header names.
+	//
+	// By default, request and response header names are normalized, i.e.
+	// The first letter and the first letters following dashes
+	// are uppercased, while all the other letters are lowercased.
+	// Examples:
+	//
+	//     * HOST -> Host
+	//     * content-type -> Content-Type
+	//     * cONTENT-lenGTH -> Content-Length
+	DisableHeaderNamesNormalizing bool
 }
 
 func (o *Options) Apply(opts []Option) {
@@ -225,6 +241,9 @@ func NewOptions(opts []Option) *Options {
 		TraceLevel: new(interface{}),
 
 		Registry: registry.NoopRegistry,
+
+		// Disabled header names' normalization, default false
+		DisableHeaderNamesNormalizing: false,
 	}
 	options.Apply(opts)
 	return options
diff --git a/pkg/protocol/header.go b/pkg/protocol/header.go
index f7533ef..14df744 100644
--- a/pkg/protocol/header.go
+++ b/pkg/protocol/header.go
@@ -1497,6 +1497,7 @@ func (h *RequestHeader) UserAgent() []byte {
 // Disable header names' normalization only if you know what are you doing.
 func (h *RequestHeader) DisableNormalizing() {
 	h.disableNormalizing = true
+	h.Trailer().DisableNormalizing()
 }
 
 func (h *RequestHeader) IsDisableNormalizing() bool {
@@ -1697,6 +1698,7 @@ func (h *RequestHeader) SetMethodBytes(method []byte) {
 // Disable header names' normalization only if you know what are you doing.
 func (h *ResponseHeader) DisableNormalizing() {
 	h.disableNormalizing = true
+	h.Trailer().DisableNormalizing()
 }
 
 // setSpecialHeader handles special headers and return true when a header is processed.
diff --git a/pkg/protocol/http1/server.go b/pkg/protocol/http1/server.go
index 77cc74e..66cb01e 100644
--- a/pkg/protocol/http1/server.go
+++ b/pkg/protocol/http1/server.go
@@ -54,20 +54,21 @@ var (
 )
 
 type Option struct {
-	StreamRequestBody            bool
-	GetOnly                      bool
-	DisablePreParseMultipartForm bool
-	DisableKeepalive             bool
-	NoDefaultServerHeader        bool
-	MaxRequestBodySize           int
-	IdleTimeout                  time.Duration
-	ReadTimeout                  time.Duration
-	ServerName                   []byte
-	TLS                          *tls.Config
-	HTMLRender                   render.HTMLRender
-	EnableTrace                  bool
-	ContinueHandler              func(header *protocol.RequestHeader) bool
-	HijackConnHandle             func(c network.Conn, h app.HijackHandler)
+	StreamRequestBody             bool
+	GetOnly                       bool
+	DisablePreParseMultipartForm  bool
+	DisableKeepalive              bool
+	NoDefaultServerHeader         bool
+	DisableHeaderNamesNormalizing bool
+	MaxRequestBodySize            int
+	IdleTimeout                   time.Duration
+	ReadTimeout                   time.Duration
+	ServerName                    []byte
+	TLS                           *tls.Config
+	HTMLRender                    render.HTMLRender
+	EnableTrace                   bool
+	ContinueHandler               func(header *protocol.RequestHeader) bool
+	HijackConnHandle              func(c network.Conn, h app.HijackHandler)
 }
 
 type Server struct {
@@ -179,6 +180,12 @@ func (s Server) Serve(c context.Context, conn network.Conn) (err error) {
 				internalStats.Record(ti, stats.ReadHeaderFinish, err)
 			})
 		}
+
+		if s.DisableHeaderNamesNormalizing {
+			ctx.Request.Header.DisableNormalizing()
+			ctx.Response.Header.DisableNormalizing()
+		}
+
 		// Read Headers
 		if err = req.ReadHeader(&ctx.Request.Header, zr); err == nil {
 			if s.EnableTrace {
diff --git a/pkg/route/engine.go b/pkg/route/engine.go
index 480e092..bd8fbc1 100644
--- a/pkg/route/engine.go
+++ b/pkg/route/engine.go
@@ -990,20 +990,21 @@ func iterate(method string, routes RoutesInfo, root *node) RoutesInfo {
 // for built-in http1 impl only.
 func newHttp1OptionFromEngine(engine *Engine) *http1.Option {
 	opt := &http1.Option{
-		StreamRequestBody:            engine.options.StreamRequestBody,
-		GetOnly:                      engine.options.GetOnly,
-		DisablePreParseMultipartForm: engine.options.DisablePreParseMultipartForm,
-		DisableKeepalive:             engine.options.DisableKeepalive,
-		NoDefaultServerHeader:        engine.options.NoDefaultServerHeader,
-		MaxRequestBodySize:           engine.options.MaxRequestBodySize,
-		IdleTimeout:                  engine.options.IdleTimeout,
-		ReadTimeout:                  engine.options.ReadTimeout,
-		ServerName:                   engine.GetServerName(),
-		ContinueHandler:              engine.ContinueHandler,
-		TLS:                          engine.options.TLS,
-		HTMLRender:                   engine.htmlRender,
-		EnableTrace:                  engine.IsTraceEnable(),
-		HijackConnHandle:             engine.HijackConnHandle,
+		StreamRequestBody:             engine.options.StreamRequestBody,
+		GetOnly:                       engine.options.GetOnly,
+		DisablePreParseMultipartForm:  engine.options.DisablePreParseMultipartForm,
+		DisableKeepalive:              engine.options.DisableKeepalive,
+		NoDefaultServerHeader:         engine.options.NoDefaultServerHeader,
+		MaxRequestBodySize:            engine.options.MaxRequestBodySize,
+		IdleTimeout:                   engine.options.IdleTimeout,
+		ReadTimeout:                   engine.options.ReadTimeout,
+		ServerName:                    engine.GetServerName(),
+		ContinueHandler:               engine.ContinueHandler,
+		TLS:                           engine.options.TLS,
+		HTMLRender:                    engine.htmlRender,
+		EnableTrace:                   engine.IsTraceEnable(),
+		HijackConnHandle:              engine.HijackConnHandle,
+		DisableHeaderNamesNormalizing: engine.options.DisableHeaderNamesNormalizing,
 	}
 	// Idle timeout of standard network must not be zero. Set it to -1 seconds if it is zero.
 	// Due to the different triggering ways of the network library, see the actual use of this value for the detailed reasons.


Test patch:
diff --git a/pkg/app/client/client_test.go b/pkg/app/client/client_test.go
index d231fac..5f3e8c6 100644
--- a/pkg/app/client/client_test.go
+++ b/pkg/app/client/client_test.go
@@ -2163,7 +2163,7 @@ func TestClientHostClientConfigHookError(t *testing.T) {
 	req.SetMethod(consts.MethodGet)
 	req.SetRequestURI("http://foo.bar/")
 	resp := protocol.AcquireResponse()
-	err := client.do(nil, req, resp)
+	err := client.do(context.TODO(), req, resp)
 	assert.DeepEqual(t, "hook return", err.Error())
 }
 
diff --git a/pkg/app/server/hertz_test.go b/pkg/app/server/hertz_test.go
index 3ce91a8..a5cf7d3 100644
--- a/pkg/app/server/hertz_test.go
+++ b/pkg/app/server/hertz_test.go
@@ -784,3 +784,39 @@ func TestSilentMode(t *testing.T) {
 		t.Fatalf("unexpected error in log: %s", b.String())
 	}
 }
+
+func TestHertzDisableHeaderNamesNormalizing(t *testing.T) {
+	h := New(
+		WithHostPorts("localhost:9212"),
+		WithDisableHeaderNamesNormalizing(true),
+	)
+	headerName := "CASE-senSITive-HEAder-NAME"
+	headerValue := "foobar-baz"
+	succeed := false
+	h.GET("/test", func(c context.Context, ctx *app.RequestContext) {
+		ctx.VisitAllHeaders(func(key, value []byte) {
+			if string(key) == headerName && string(value) == headerValue {
+				succeed = true
+				return
+			}
+		})
+		if !succeed {
+			t.Fatalf("DisableHeaderNamesNormalizing failed")
+		} else {
+			ctx.Header(headerName, headerValue)
+		}
+	})
+
+	go h.Spin()
+	time.Sleep(100 * time.Millisecond)
+
+	cli, _ := c.NewClient(c.WithDisableHeaderNamesNormalizing(true))
+
+	r := protocol.NewRequest("GET", "http://localhost:9212/test", nil)
+	r.Header.DisableNormalizing()
+	r.Header.Set(headerName, headerValue)
+	res := protocol.AcquireResponse()
+	err := cli.Do(context.Background(), r, res)
+	assert.Nil(t, err)
+	assert.DeepEqual(t, headerValue, res.Header.Get(headerName))
+}
diff --git a/pkg/app/server/option_test.go b/pkg/app/server/option_test.go
index aef554c..f5d7f7b 100644
--- a/pkg/app/server/option_test.go
+++ b/pkg/app/server/option_test.go
@@ -75,6 +75,7 @@ func TestOptions(t *testing.T) {
 		WithAutoReloadRender(true, 5*time.Second),
 		WithListenConfig(cfg),
 		WithAltTransport(transporter),
+		WithDisableHeaderNamesNormalizing(true),
 	})
 	assert.DeepEqual(t, opt.ReadTimeout, time.Second)
 	assert.DeepEqual(t, opt.WriteTimeout, time.Second)
@@ -107,6 +108,7 @@ func TestOptions(t *testing.T) {
 	assert.DeepEqual(t, opt.AutoReloadInterval, 5*time.Second)
 	assert.DeepEqual(t, opt.ListenConfig, cfg)
 	assert.Assert(t, reflect.TypeOf(opt.AltTransporterNewer) == reflect.TypeOf(transporter))
+	assert.DeepEqual(t, opt.DisableHeaderNamesNormalizing, true)
 }
 
 func TestDefaultOptions(t *testing.T) {
@@ -139,6 +141,7 @@ func TestDefaultOptions(t *testing.T) {
 	assert.Assert(t, opt.RegistryInfo == nil)
 	assert.DeepEqual(t, opt.AutoReloadRender, false)
 	assert.DeepEqual(t, opt.AutoReloadInterval, time.Duration(0))
+	assert.DeepEqual(t, opt.DisableHeaderNamesNormalizing, false)
 }
 
 type mockTransporter struct{}
diff --git a/pkg/common/config/option_test.go b/pkg/common/config/option_test.go
index 49ea661..39d92d7 100644
--- a/pkg/common/config/option_test.go
+++ b/pkg/common/config/option_test.go
@@ -53,6 +53,7 @@ func TestDefaultOptions(t *testing.T) {
 	assert.DeepEqual(t, []interface{}{}, options.Tracers)
 	assert.DeepEqual(t, new(interface{}), options.TraceLevel)
 	assert.DeepEqual(t, registry.NoopRegistry, options.Registry)
+	assert.DeepEqual(t, false, options.DisableHeaderNamesNormalizing)
 }
 
 // TestApplyCustomOptions test apply options with custom values after init