Repository: go-resty/resty
Pull request #549 (2022-06-15 10:37:49)

Fix multipart file readers not being reset when doing a retry
As mentioned in https://github.com/go-resty/resty/issues/166 by @neganovalexey, when doing a retry on a multipart request the `io.Reader` is not reset to the start again. This can lead to subtle bugs, as the request will be retried, but the form part of the file will be empty, because the reader was already exhausted. Solving this issue in the caller code is pretty ugly, because `Request` does not expose access to `multipartFiles` and so the caller would have to use `RetryHooks` and recreate the `Response.Request` and that at each call site (where there is access to the original `ReadSeeker`).

In the PR, I created a new client options `SetRetryResetReaders` which adds a retry hook  seeking the start of each reader in `multipartFiles`, if the reader it got also implements `io.ReadSeeker`. This solved the above issue described above for my usecase and should have minimal implications on existing usages.

SVilgelm: What about the readers that cannot be reset?
nikplx: @SVilgelm 
They won't be reset. The code only resets the reader if it implements `io.ReadSeeker` and that only if you enable it, so as to not break any running code. Every other reader will be untouched. I don't know if there is a usecase for passing in a reader that should not be reset, but it will not do anything if there is a reader that can not be reset.
jeevatkm: @nikplx I'm sorry for the delayed attention on the PR. Thanks for your contribution.

jeevatkm: @nikplx I'm sorry for the delayed attention on the PR. Thanks for your contribution.
Yes, it make sense to provide the reset readers option for resty users.
nikplx: Hey @jeevatkm 

I fixed the test stage. There was some mixup in the sum file. It now runs locally. Maybe you can retrigger the pipeline if you have some time.
codecov[bot]: # [Codecov](https://codecov.io/gh/go-resty/resty/pull/549?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty) Report
> Merging [#549](https://codecov.io/gh/go-resty/resty/pull/549?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty) (7990f74) into [master](https://codecov.io/gh/go-resty/resty/commit/0451c4c63033b1a330b890cd543c97501b8684b9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty) (0451c4c) will **decrease** coverage by `0.25%`.
> The diff coverage is `73.33%`.

```diff
@@            Coverage Diff             @@
##           master     #549      +/-   ##
==========================================
- Coverage   95.94%   95.69%   -0.25%     
==========================================
  Files          10       10              
  Lines        1357     1372      +15     
==========================================
+ Hits         1302     1313      +11     
- Misses         34       36       +2     
- Partials       21       23       +2     
```


| [Impacted Files](https://codecov.io/gh/go-resty/resty/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty) | Coverage Δ | |
|---|---|---|
| [request.go](https://codecov.io/gh/go-resty/resty/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty#diff-cmVxdWVzdC5nbw==) | `95.47% <ø> (ø)` | |
| [retry.go](https://codecov.io/gh/go-resty/resty/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty#diff-cmV0cnkuZ28=) | `96.22% <66.66%> (-3.78%)` | :arrow_down: |
| [client.go](https://codecov.io/gh/go-resty/resty/pull/549?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty#diff-Y2xpZW50Lmdv) | `97.78% <100.00%> (+0.01%)` | :arrow_up: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=go-resty)

jeevatkm: @nikplx This time PR validation build executed without any issue.

Do you mind checking this [codecov feedback](https://github.com/go-resty/resty/pull/549/checks?check_run_id=11786008284)? 

Patch:
diff --git a/client.go b/client.go
index f8823e8..326a4a5 100644
--- a/client.go
+++ b/client.go
@@ -116,6 +116,7 @@ type Client struct {
 	RetryConditions       []RetryConditionFunc
 	RetryHooks            []OnRetryFunc
 	RetryAfter            RetryAfterFunc
+	RetryResetReaders     bool
 	JSONMarshal           func(v interface{}) ([]byte, error)
 	JSONUnmarshal         func(data []byte, v interface{}) error
 	XMLMarshal            func(v interface{}) ([]byte, error)
@@ -702,6 +703,15 @@ func (c *Client) AddRetryHook(hook OnRetryFunc) *Client {
 	return c
 }
 
+// SetRetryResetReaders method enables the Resty client to seek the start of all
+// file readers given as multipart files, if the given object implements `io.ReadSeeker`.
+//
+// Since ...
+func (c *Client) SetRetryResetReaders(b bool) *Client {
+	c.RetryResetReaders = b
+	return c
+}
+
 // SetTLSClientConfig method sets TLSClientConfig for underling client Transport.
 //
 // For Example:
diff --git a/go.mod b/go.mod
index 5e78bdc..d712b6e 100644
--- a/go.mod
+++ b/go.mod
@@ -1,5 +1,5 @@
 module github.com/go-resty/resty/v2
 
-require golang.org/x/net v0.0.0-20211029224645-99673261e6eb
+require golang.org/x/net v0.0.0-20220325170049-de3da57026de
 
 go 1.11
diff --git a/go.sum b/go.sum
index 07a4515..7b4eb43 100644
--- a/go.sum
+++ b/go.sum
@@ -1,7 +1,7 @@
-golang.org/x/net v0.0.0-20211029224645-99673261e6eb h1:pirldcYWx7rx7kE5r+9WsOXPXK0+WH5+uZ7uPmJ44uM=
-golang.org/x/net v0.0.0-20211029224645-99673261e6eb/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
-golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/net v0.0.0-20220325170049-de3da57026de h1:pZB1TWnKi+o4bENlbzAgLrEbY4RMYmUIRobMcSmfeYc=
+golang.org/x/net v0.0.0-20220325170049-de3da57026de/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
diff --git a/request.go b/request.go
index 25b4c3b..adb129e 100644
--- a/request.go
+++ b/request.go
@@ -847,6 +847,7 @@ func (r *Request) Execute(method, url string) (*Response, error) {
 		MaxWaitTime(r.client.RetryMaxWaitTime),
 		RetryConditions(append(r.retryConditions, r.client.RetryConditions...)),
 		RetryHooks(r.client.RetryHooks),
+		ResetMultipartReaders(r.client.RetryResetReaders),
 	)
 
 	r.client.onErrorHooks(r, resp, unwrapNoRetryErr(err))
diff --git a/retry.go b/retry.go
index 00b8514..f8c7665 100644
--- a/retry.go
+++ b/retry.go
@@ -6,6 +6,7 @@ package resty
 
 import (
 	"context"
+	"io"
 	"math"
 	"math/rand"
 	"sync"
@@ -43,6 +44,7 @@ type (
 		maxWaitTime     time.Duration
 		retryConditions []RetryConditionFunc
 		retryHooks      []OnRetryFunc
+		resetReaders    bool
 	}
 )
 
@@ -81,6 +83,14 @@ func RetryHooks(hooks []OnRetryFunc) Option {
 	}
 }
 
+// ResetMultipartReaders sets a boolean value which will lead the start being seeked out
+// on all multipart file readers, if they implement io.ReadSeeker
+func ResetMultipartReaders(value bool) Option {
+	return func(o *Options) {
+		o.resetReaders = value
+	}
+}
+
 // Backoff retries with increasing timeout duration up until X amount of retries
 // (Default is 3 attempts, Override with option Retries(n))
 func Backoff(operation func() (*Response, error), options ...Option) error {
@@ -125,6 +135,12 @@ func Backoff(operation func() (*Response, error), options ...Option) error {
 			return err
 		}
 
+		if opts.resetReaders {
+			if err := resetFileReaders(resp.Request.multipartFiles); err != nil {
+				return err
+			}
+		}
+
 		for _, hook := range opts.retryHooks {
 			hook(resp, err)
 		}
@@ -219,3 +235,15 @@ func newRnd() *rand.Rand {
 	var src = rand.NewSource(seed)
 	return rand.New(src)
 }
+
+func resetFileReaders(files []*File) error {
+	for _, f := range files {
+		if rs, ok := f.Reader.(io.ReadSeeker); ok {
+			if _, err := rs.Seek(0, io.SeekStart); err != nil {
+				return err
+			}
+		}
+	}
+
+	return nil
+}


Test patch:
diff --git a/resty_test.go b/resty_test.go
index b815418..b97e066 100644
--- a/resty_test.go
+++ b/resty_test.go
@@ -441,6 +441,10 @@ func createFilePostServer(t *testing.T) *httptest.Server {
 			size, _ := io.Copy(f, r.Body)
 
 			fmt.Fprintf(w, "File Uploaded successfully, file size: %v", size)
+		case "/set-reset-multipart-readers-test":
+			w.Header().Set(hdrContentTypeKey, "application/json; charset=utf-8")
+			w.WriteHeader(http.StatusInternalServerError)
+			_, _ = fmt.Fprintf(w, `{ "message": "error" }`)
 		}
 	})
 
diff --git a/retry_test.go b/retry_test.go
index 9f8fb38..d979fbf 100644
--- a/retry_test.go
+++ b/retry_test.go
@@ -5,9 +5,12 @@
 package resty
 
 import (
+	"bytes"
 	"context"
 	"encoding/json"
 	"errors"
+	"fmt"
+	"io"
 	"net/http"
 	"reflect"
 	"strconv"
@@ -55,7 +58,7 @@ func TestBackoffNoWaitForLastRetry(t *testing.T) {
 		externalCounter++
 		return resp, nil
 	}, RetryConditions([]RetryConditionFunc{func(response *Response, err error) bool {
-		if externalCounter == attempts + numRetries {
+		if externalCounter == attempts+numRetries {
 			// Backoff returns context canceled if goes to sleep after last retry.
 			cancel()
 		}
@@ -733,3 +736,76 @@ func TestClientRetryHook(t *testing.T) {
 func filler(*Response, error) bool {
 	return false
 }
+
+var seekFailure = fmt.Errorf("failing seek test")
+
+type failingSeeker struct {
+	reader *bytes.Reader
+}
+
+func (f failingSeeker) Read(b []byte) (n int, err error) {
+	return f.reader.Read(b)
+}
+
+func (f failingSeeker) Seek(offset int64, whence int) (int64, error) {
+	if offset == 0 && whence == io.SeekStart {
+		return 0, seekFailure
+	}
+
+	return f.reader.Seek(offset, whence)
+}
+
+func TestResetMultipartReaderSeekStartError(t *testing.T) {
+	ts := createFilePostServer(t)
+	defer ts.Close()
+
+	testSeeker := &failingSeeker{
+		bytes.NewReader([]byte("test")),
+	}
+
+	c := dc().
+		SetRetryCount(2).
+		SetTimeout(time.Second * 3).
+		SetRetryResetReaders(true).
+		AddRetryAfterErrorCondition()
+
+	resp, err := c.R().
+		SetFileReader("name", "filename", testSeeker).
+		Post(ts.URL + "/set-reset-multipart-readers-test")
+
+	assertEqual(t, 500, resp.StatusCode())
+	assertEqual(t, err.Error(), seekFailure.Error())
+}
+
+func TestResetMultipartReaders(t *testing.T) {
+	ts := createFilePostServer(t)
+	defer ts.Close()
+
+	str := "test"
+	buf := []byte(str)
+
+	bufReader := bytes.NewReader(buf)
+	bufCpy := make([]byte, len(buf))
+
+	c := dc().
+		SetRetryCount(2).
+		SetTimeout(time.Second * 3).
+		SetRetryResetReaders(true).
+		AddRetryAfterErrorCondition().
+		AddRetryHook(
+			func(response *Response, _ error) {
+				read, err := bufReader.Read(bufCpy)
+
+				assertNil(t, err)
+				assertEqual(t, len(buf), read)
+				assertEqual(t, str, string(bufCpy))
+			},
+		)
+
+	resp, err := c.R().
+		SetFileReader("name", "filename", bufReader).
+		Post(ts.URL + "/set-reset-multipart-readers-test")
+
+	assertEqual(t, 500, resp.StatusCode())
+	assertNil(t, err)
+}