Repository: milvus-io/milvus-sdk-go
Pull request #406 (2023-02-22 04:07:17)

add IgnoreGrowing option
related: #407 https://github.com/milvus-io/milvus/issues/21836
Signed-off-by: aoiasd [zhicheng.yue@zilliz.com](mailto:zhicheng.yue@zilliz.com)

sre-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/milvus-io/milvus-sdk-go/pull/406#" title="Author self-approved">aoiasd</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **sunby** after the PR has been reviewed.
You can assign the PR to them by writing `/assign @sunby` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=milvus-io%2Fmilvus-sdk-go).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/milvus-io/milvus-sdk-go/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["sunby"]} -->
mergify[bot]: @aoiasd Please associate the related issue to the body of your Pull Request.  (eg. “issue: #187”)
codecov[bot]: # [Codecov](https://codecov.io/gh/milvus-io/milvus-sdk-go/pull/406?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) Report
> Merging [#406](https://codecov.io/gh/milvus-io/milvus-sdk-go/pull/406?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) (63cc4ea) into [master](https://codecov.io/gh/milvus-io/milvus-sdk-go/commit/f5d28d7b2a2b174c5e930355f40396028857f911?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) (f5d28d7) will **decrease** coverage by `0.10%`.
> The diff coverage is `66.66%`.

[![Impacted file tree graph](https://codecov.io/gh/milvus-io/milvus-sdk-go/pull/406/graphs/tree.svg?width=650&height=150&src=pr&token=WJK43PV390&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io)](https://codecov.io/gh/milvus-io/milvus-sdk-go/pull/406?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io)

```diff
@@            Coverage Diff             @@
##           master     #406      +/-   ##
==========================================
- Coverage   83.40%   83.30%   -0.10%     
==========================================
  Files          30       30              
  Lines        3049     3055       +6     
==========================================
+ Hits         2543     2545       +2     
- Misses        413      416       +3     
- Partials       93       94       +1     
```


| [Impacted Files](https://codecov.io/gh/milvus-io/milvus-sdk-go/pull/406?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | Coverage Δ | |
|---|---|---|
| [client/client\_grpc\_options.go](https://codecov.io/gh/milvus-io/milvus-sdk-go/pull/406?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io#diff-Y2xpZW50L2NsaWVudF9ncnBjX29wdGlvbnMuZ28=) | `75.00% <0.00%> (-3.27%)` | :arrow_down: |
| [client/client\_grpc\_data.go](https://codecov.io/gh/milvus-io/milvus-sdk-go/pull/406?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io#diff-Y2xpZW50L2NsaWVudF9ncnBjX2RhdGEuZ28=) | `72.14% <80.00%> (-0.18%)` | :arrow_down: |

congqixia: /kind enhancement
sre-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/milvus-io/milvus-sdk-go/pull/406#" title="Author self-approved">aoiasd</a>*, *<a href="https://github.com/milvus-io/milvus-sdk-go/pull/406#pullrequestreview-1316732380" title="LGTM">congqixia</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=milvus-io%2Fmilvus-sdk-go).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/milvus-io/milvus-sdk-go/blob/master/OWNERS)~~ [congqixia]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
sre-ci-robot: New changes are detected. LGTM label has been removed.
sre-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/milvus-io/milvus-sdk-go/pull/406#" title="Author self-approved">aoiasd</a>*, *<a href="https://github.com/milvus-io/milvus-sdk-go/pull/406#pullrequestreview-1327543035" title="LGTM">congqixia</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=milvus-io%2Fmilvus-sdk-go).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/milvus-io/milvus-sdk-go/blob/master/OWNERS)~~ [congqixia]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->

Patch:
diff --git a/client/client_grpc_data.go b/client/client_grpc_data.go
index 3befd64..59ca04f 100644
--- a/client/client_grpc_data.go
+++ b/client/client_grpc_data.go
@@ -30,8 +30,9 @@ import (
 )
 
 const (
-	offsetKey = `offset`
-	limitKey  = `limit`
+	offsetKey        = `offset`
+	limitKey         = `limit`
+	ignoreGrowingKey = `ignore_growing`
 )
 
 // Insert Index  into collection with column-based format
@@ -359,6 +360,9 @@ func (c *GrpcClient) Query(ctx context.Context, collectionName string, partition
 	if option.Limit > 0 {
 		req.QueryParams = append(req.QueryParams, &common.KeyValuePair{Key: limitKey, Value: strconv.FormatInt(option.Limit, 10)})
 	}
+	if option.IgnoreGrowing {
+		req.QueryParams = append(req.QueryParams, &common.KeyValuePair{Key: ignoreGrowingKey, Value: strconv.FormatBool(option.IgnoreGrowing)})
+	}
 
 	resp, err := c.Service.Query(ctx, req)
 	if err != nil {
@@ -408,13 +412,15 @@ func prepareSearchRequest(collName string, partitions []string,
 	if err != nil {
 		return nil, err
 	}
+
 	searchParams := entity.MapKvPairs(map[string]string{
-		"anns_field":    vectorField,
-		"topk":          fmt.Sprintf("%d", topK),
-		"params":        string(bs),
-		"metric_type":   string(metricType),
-		"round_decimal": "-1",
-		offsetKey:       fmt.Sprintf("%d", opt.Offset),
+		"anns_field":     vectorField,
+		"topk":           fmt.Sprintf("%d", topK),
+		"params":         string(bs),
+		"metric_type":    string(metricType),
+		"round_decimal":  "-1",
+		ignoreGrowingKey: strconv.FormatBool(opt.IgnoreGrowing),
+		offsetKey:        fmt.Sprintf("%d", opt.Offset),
 	})
 	req := &server.SearchRequest{
 		DbName:             "",
diff --git a/client/client_grpc_options.go b/client/client_grpc_options.go
index f856d89..b1fa0b5 100644
--- a/client/client_grpc_options.go
+++ b/client/client_grpc_options.go
@@ -56,11 +56,19 @@ type SearchQueryOption struct {
 	// Pagination
 	Limit  int64
 	Offset int64
+
+	IgnoreGrowing bool
 }
 
 // SearchQueryOptionFunc is a function which modifies SearchOption
 type SearchQueryOptionFunc func(option *SearchQueryOption)
 
+func WithIgnoreGrowing() SearchQueryOptionFunc {
+	return func(option *SearchQueryOption) {
+		option.IgnoreGrowing = true
+	}
+}
+
 // WithOffset returns search/query option with offset.
 func WithOffset(offset int64) SearchQueryOptionFunc {
 	return func(option *SearchQueryOption) {


Test patch:
diff --git a/client/client_grpc_data_test.go b/client/client_grpc_data_test.go
index baeac8e..56fce5f 100644
--- a/client/client_grpc_data_test.go
+++ b/client/client_grpc_data_test.go
@@ -338,6 +338,12 @@ func TestGrpcSearch(t *testing.T) {
 		assert.Nil(t, err)
 		assert.NotNil(t, results)
 
+		// search with ignore growing
+		results, err = c.Search(ctx, testCollectionName, []string{}, expr, []string{"int64"}, []entity.Vector{entity.FloatVector(vectors[0])},
+			testVectorField, entity.L2, 10, sp, WithIgnoreGrowing())
+		assert.Nil(t, err)
+		assert.NotNil(t, results)
+
 		// search with session consistency level
 		results, err = c.Search(ctx, testCollectionName, []string{}, expr, []string{"int64"}, []entity.Vector{entity.FloatVector(vectors[0])},
 			testVectorField, entity.L2, 10, sp, WithSearchQueryConsistencyLevel(entity.ClSession), WithTravelTimestamp(0))
@@ -431,6 +437,18 @@ func TestGrpcQuery(t *testing.T) {
 		colInt64, ok = columns[0].(*entity.ColumnInt64)
 		assert.True(t, ok)
 		assert.ElementsMatch(t, []int64{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, colInt64.Data())
+
+		// query with ignore growing options
+		columns, err = c.Query(ctx, testCollectionName, []string{partName}, "int64 in {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}", []string{"int64", testVectorField}, WithIgnoreGrowing())
+		assert.NoError(t, err)
+		assert.Equal(t, 2, len(columns))
+		assert.Equal(t, entity.FieldTypeInt64, columns[0].Type())
+		assert.Equal(t, entity.FieldTypeFloatVector, columns[1].Type())
+		assert.Equal(t, 10, columns[0].Len())
+
+		colInt64, ok = columns[0].(*entity.ColumnInt64)
+		assert.True(t, ok)
+		assert.ElementsMatch(t, []int64{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, colInt64.Data())
 	})
 
 	t.Run("normal query varchar pks", func(t *testing.T) {
diff --git a/client/client_grpc_options_test.go b/client/client_grpc_options_test.go
index d64a165..b876238 100644
--- a/client/client_grpc_options_test.go
+++ b/client/client_grpc_options_test.go
@@ -63,6 +63,19 @@ func TestMakeSearchQueryOption(t *testing.T) {
 		expected := &SearchQueryOption{
 			ConsistencyLevel:   entity.ClStrong,
 			GuaranteeTimestamp: StrongTimestamp,
+			IgnoreGrowing:      false,
+		}
+		assert.Equal(t, expected, opt)
+	})
+
+	t.Run("ignore growing", func(t *testing.T) {
+		opt, err := makeSearchQueryOption(c.Name, WithIgnoreGrowing())
+		assert.Nil(t, err)
+		assert.NotNil(t, opt)
+		expected := &SearchQueryOption{
+			ConsistencyLevel:   entity.ClStrong,
+			GuaranteeTimestamp: StrongTimestamp,
+			IgnoreGrowing:      true,
 		}
 		assert.Equal(t, expected, opt)
 	})
@@ -74,6 +87,7 @@ func TestMakeSearchQueryOption(t *testing.T) {
 		expected := &SearchQueryOption{
 			ConsistencyLevel:   entity.ClSession,
 			GuaranteeTimestamp: EventuallyTimestamp,
+			IgnoreGrowing:      false,
 		}
 		assert.Equal(t, expected, opt)
 
@@ -84,6 +98,7 @@ func TestMakeSearchQueryOption(t *testing.T) {
 		expected = &SearchQueryOption{
 			ConsistencyLevel:   entity.ClSession,
 			GuaranteeTimestamp: 99,
+			IgnoreGrowing:      false,
 		}
 		assert.Equal(t, expected, opt)
 	})
@@ -95,6 +110,7 @@ func TestMakeSearchQueryOption(t *testing.T) {
 		expected := &SearchQueryOption{
 			ConsistencyLevel:   entity.ClBounded,
 			GuaranteeTimestamp: BoundedTimestamp,
+			IgnoreGrowing:      false,
 		}
 		assert.Equal(t, expected, opt)
 	})
@@ -106,6 +122,7 @@ func TestMakeSearchQueryOption(t *testing.T) {
 		expected := &SearchQueryOption{
 			ConsistencyLevel:   entity.ClEventually,
 			GuaranteeTimestamp: EventuallyTimestamp,
+			IgnoreGrowing:      false,
 		}
 		assert.Equal(t, expected, opt)
 	})
@@ -117,6 +134,7 @@ func TestMakeSearchQueryOption(t *testing.T) {
 		expected := &SearchQueryOption{
 			ConsistencyLevel:   entity.ClCustomized,
 			GuaranteeTimestamp: 100,
+			IgnoreGrowing:      false,
 		}
 		assert.Equal(t, expected, opt)
 	})