Repository: kopia/kopia
Pull request #2623 (2022-12-03 23:06:20)

feat(repository): added OnSuccessfulFlush()

codecov[bot]: # [Codecov](https://codecov.io/gh/kopia/kopia/pull/2623?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia) Report
Base: **74.86**% // Head: **74.86**% // Increases project coverage by **`+0.00%`** :tada:
> Coverage data is based on head [(`d2ac24c`)](https://codecov.io/gh/kopia/kopia/pull/2623?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia) compared to base [(`5463fb7`)](https://codecov.io/gh/kopia/kopia/commit/5463fb7e1b43af16ae092a60e6ec9cf2ad2a2b59?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master    #2623   +/-   ##
=======================================
  Coverage   74.86%   74.86%           
=======================================
  Files         453      453           
  Lines       35581    35586    +5     
=======================================
+ Hits        26639    26643    +4     
- Misses       7045     7047    +2     
+ Partials     1897     1896    -1     
```


| [Impacted Files](https://codecov.io/gh/kopia/kopia/pull/2623?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia) | Coverage Δ | |
|---|---|---|
| [repo/open.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-cmVwby9vcGVuLmdv) | `79.33% <ø> (-0.14%)` | :arrow_down: |
| [repo/server\_repository\_params.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-cmVwby9zZXJ2ZXJfcmVwb3NpdG9yeV9wYXJhbXMuZ28=) | `50.00% <ø> (ø)` | |
| [repo/api\_server\_repository.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-cmVwby9hcGlfc2VydmVyX3JlcG9zaXRvcnkuZ28=) | `64.65% <100.00%> (+0.50%)` | :arrow_up: |
| [repo/grpc\_repository\_client.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-cmVwby9ncnBjX3JlcG9zaXRvcnlfY2xpZW50Lmdv) | `75.55% <100.00%> (+0.42%)` | :arrow_up: |
| [repo/repository.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-cmVwby9yZXBvc2l0b3J5Lmdv) | `80.48% <100.00%> (+0.24%)` | :arrow_up: |
| [cli/command\_content\_verify.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-Y2xpL2NvbW1hbmRfY29udGVudF92ZXJpZnkuZ28=) | `75.63% <0.00%> (-5.89%)` | :arrow_down: |
| [snapshot/snapshotfs/repofs.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-c25hcHNob3Qvc25hcHNob3Rmcy9yZXBvZnMuZ28=) | `79.29% <0.00%> (-1.52%)` | :arrow_down: |
| [fs/ignorefs/ignorefs.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-ZnMvaWdub3JlZnMvaWdub3JlZnMuZ28=) | `77.27% <0.00%> (-1.24%)` | :arrow_down: |
| [repo/content/index/index\_v1.go](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia#diff-cmVwby9jb250ZW50L2luZGV4L2luZGV4X3YxLmdv) | `75.00% <0.00%> (-1.18%)` | :arrow_down: |
| ... and [3 more](https://codecov.io/gh/kopia/kopia/pull/2623/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia) | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/kopia/kopia/pull/2623?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=kopia).


Patch:
diff --git a/repo/api_server_repository.go b/repo/api_server_repository.go
index 520dbc9d..e0e1b38c 100644
--- a/repo/api_server_repository.go
+++ b/repo/api_server_repository.go
@@ -34,6 +34,7 @@ type apiServerRepository struct {
 	serverSupportsContentCompression bool
 	omgr                             *object.Manager
 	wso                              WriteSessionOptions
+	afterFlush                       []RepositoryWriterCallback
 
 	*immutableServerRepositoryParameters // immutable parameters
 }
@@ -164,6 +165,7 @@ func (r *apiServerRepository) NewWriter(ctx context.Context, opt WriteSessionOpt
 
 	w.omgr = omgr
 	w.wso = opt
+	w.afterFlush = nil
 
 	if w.wso.OnUpload == nil {
 		w.wso.OnUpload = func(i int64) {}
@@ -271,6 +273,11 @@ func (r *apiServerRepository) PrefetchContents(ctx context.Context, contentIDs [
 	return resp.ContentIDs
 }
 
+// OnSuccessfulFlush registers the provided callback to be invoked after flush succeeds.
+func (r *apiServerRepository) OnSuccessfulFlush(callback RepositoryWriterCallback) {
+	r.afterFlush = append(r.afterFlush, callback)
+}
+
 var _ Repository = (*apiServerRepository)(nil)
 
 // openRestAPIRepository connects remote repository over Kopia API.
diff --git a/repo/grpc_repository_client.go b/repo/grpc_repository_client.go
index 20994a84..ae088a92 100644
--- a/repo/grpc_repository_client.go
+++ b/repo/grpc_repository_client.go
@@ -67,6 +67,8 @@ type grpcRepositoryClient struct {
 	isReadOnly         bool
 	transparentRetries bool
 
+	afterFlush []RepositoryWriterCallback
+
 	// how many times we tried to establish inner session
 	// +checklocks:innerSessionMutex
 	innerSessionAttemptCount int
@@ -726,6 +728,11 @@ func (r *grpcRepositoryClient) UpdateDescription(d string) {
 	r.cliOpts.Description = d
 }
 
+// OnSuccessfulFlush registers the provided callback to be invoked after flush succeeds.
+func (r *grpcRepositoryClient) OnSuccessfulFlush(callback RepositoryWriterCallback) {
+	r.afterFlush = append(r.afterFlush, callback)
+}
+
 var _ Repository = (*grpcRepositoryClient)(nil)
 
 type grpcCreds struct {
diff --git a/repo/open.go b/repo/open.go
index 888325f1..f691a7ee 100644
--- a/repo/open.go
+++ b/repo/open.go
@@ -65,13 +65,12 @@
 
 // Options provides configuration parameters for connection to a repository.
 type Options struct {
-	TraceStorage        bool                        // Logs all storage access using provided Printf-style function
-	TimeNowFunc         func() time.Time            // Time provider
-	DisableInternalLog  bool                        // Disable internal log
-	UpgradeOwnerID      string                      // Owner-ID of any upgrade in progress, when this is not set the access may be restricted
-	DoNotWaitForUpgrade bool                        // Disable the exponential forever backoff on an upgrade lock.
-	BeforeFlush         []RepositoryWriterCallbacks // list of callbacks to invoke before every flush
-	AfterFlush          []RepositoryWriterCallbacks // list of callbacks to invoke after every flush
+	TraceStorage        bool                       // Logs all storage access using provided Printf-style function
+	TimeNowFunc         func() time.Time           // Time provider
+	DisableInternalLog  bool                       // Disable internal log
+	UpgradeOwnerID      string                     // Owner-ID of any upgrade in progress, when this is not set the access may be restricted
+	DoNotWaitForUpgrade bool                       // Disable the exponential forever backoff on an upgrade lock.
+	BeforeFlush         []RepositoryWriterCallback // list of callbacks to invoke before every flush
 
 	OnFatalError func(err error) // function to invoke when repository encounters a fatal error, usually invokes os.Exit
 
@@ -190,7 +189,6 @@ func(ctx context.Context) error {
 		metricsRegistry:  mr,
 		refCountedCloser: closer,
 		beforeFlush:      options.BeforeFlush,
-		afterFlush:       options.AfterFlush,
 	}
 
 	if si.DisableGRPC {
@@ -353,7 +351,6 @@ func openWithConfig(ctx context.Context, st blob.Storage, cliOpts ClientOptions,
 			metricsRegistry:  mr,
 			refCountedCloser: closer,
 			beforeFlush:      options.BeforeFlush,
-			afterFlush:       options.AfterFlush,
 		},
 	}
 
diff --git a/repo/repository.go b/repo/repository.go
index 58105eff..77e1d2e2 100644
--- a/repo/repository.go
+++ b/repo/repository.go
@@ -49,6 +49,7 @@ type RepositoryWriter interface {
 	ConcatenateObjects(ctx context.Context, objectIDs []object.ID) (object.ID, error)
 	PutManifest(ctx context.Context, labels map[string]string, payload interface{}) (manifest.ID, error)
 	DeleteManifest(ctx context.Context, id manifest.ID) error
+	OnSuccessfulFlush(callback RepositoryWriterCallback)
 	Flush(ctx context.Context) error
 }
 
@@ -97,16 +98,15 @@ type immutableDirectRepositoryParameters struct {
 	nextWriterID    *int32
 	throttler       throttling.SettableThrottler
 	metricsRegistry *metrics.Registry
-	beforeFlush     []RepositoryWriterCallbacks
-	afterFlush      []RepositoryWriterCallbacks
+	beforeFlush     []RepositoryWriterCallback
 
 	*refCountedCloser
 }
 
-// RepositoryWriterCallbacks is a hook function invoked before and after each flush.
-type RepositoryWriterCallbacks func(ctx context.Context, w RepositoryWriter) error
+// RepositoryWriterCallback is a hook function invoked before and after each flush.
+type RepositoryWriterCallback func(ctx context.Context, w RepositoryWriter) error
 
-func invokeCallbacks(ctx context.Context, w RepositoryWriter, callbacks []RepositoryWriterCallbacks) error {
+func invokeCallbacks(ctx context.Context, w RepositoryWriter, callbacks []RepositoryWriterCallback) error {
 	for _, h := range callbacks {
 		if err := h(ctx, w); err != nil {
 			return err
@@ -125,6 +125,8 @@ type directRepository struct {
 	omgr  *object.Manager
 	mmgr  *manifest.Manager
 	sm    *content.SharedManager
+
+	afterFlush []RepositoryWriterCallback
 }
 
 // DeriveKey derives encryption key of the provided length from the master key.
@@ -362,6 +364,11 @@ func (r *directRepository) FormatManager() *format.Manager {
 	return r.fmgr
 }
 
+// OnSuccessfulFlush registers the provided callback to be invoked after flush succeeds.
+func (r *directRepository) OnSuccessfulFlush(callback RepositoryWriterCallback) {
+	r.afterFlush = append(r.afterFlush, callback)
+}
+
 // WriteSessionOptions describes options for a write session.
 type WriteSessionOptions struct {
 	Purpose        string
diff --git a/repo/server_repository_params.go b/repo/server_repository_params.go
index 5cb7e8f1..93ef1a77 100644
--- a/repo/server_repository_params.go
+++ b/repo/server_repository_params.go
@@ -14,8 +14,7 @@ type immutableServerRepositoryParameters struct {
 	cliOpts         ClientOptions
 	metricsRegistry *metrics.Registry
 	contentCache    *cache.PersistentCache
-	beforeFlush     []RepositoryWriterCallbacks
-	afterFlush      []RepositoryWriterCallbacks
+	beforeFlush     []RepositoryWriterCallback
 
 	*refCountedCloser
 }


Test patch:
diff --git a/repo/repository_test.go b/repo/repository_test.go
index 37f46255..72de1dbc 100644
--- a/repo/repository_test.go
+++ b/repo/repository_test.go
@@ -593,11 +593,10 @@ func TestWriteSessionFlushOnSuccess(t *testing.T) {
 		OpenOptions: func(o *repo.Options) {
 			o.BeforeFlush = append(o.BeforeFlush, func(ctx context.Context, w repo.RepositoryWriter) error {
 				beforeFlushCount.Add(1)
-				return nil
-			})
-
-			o.AfterFlush = append(o.AfterFlush, func(ctx context.Context, w repo.RepositoryWriter) error {
-				afterFlushCount.Add(1)
+				w.OnSuccessfulFlush(func(ctx context.Context, w repo.RepositoryWriter) error {
+					afterFlushCount.Add(1)
+					return nil
+				})
 				return nil
 			})
 		},
@@ -662,15 +661,13 @@ func testWriteSessionFlushOnSuccessClient(t *testing.T, disableGRPC bool) {
 	}, content.CachingOptions{
 		CacheDirectory: testutil.TempDirectory(t),
 	}, servertesting.TestPassword, &repo.Options{
-		BeforeFlush: []repo.RepositoryWriterCallbacks{
+		BeforeFlush: []repo.RepositoryWriterCallback{
 			func(ctx context.Context, w repo.RepositoryWriter) error {
 				beforeFlushCount.Add(1)
-				return nil
-			},
-		},
-		AfterFlush: []repo.RepositoryWriterCallbacks{
-			func(ctx context.Context, w repo.RepositoryWriter) error {
-				afterFlushCount.Add(1)
+				w.OnSuccessfulFlush(func(ctx context.Context, w repo.RepositoryWriter) error {
+					afterFlushCount.Add(1)
+					return nil
+				})
 				return nil
 			},
 		},