Repository: oom-ai/oomstore
Pull request #845 (2021-12-29 07:11:16)

ci: add docker cache

codecov[bot]: # [Codecov](https://codecov.io/gh/oom-ai/oomstore/pull/845?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai) Report
> Merging [#845](https://codecov.io/gh/oom-ai/oomstore/pull/845?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai) (71152d9) into [main](https://codecov.io/gh/oom-ai/oomstore/commit/1a80e63d0165b7281d39cf1f1f83038d6604bba5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai) (1a80e63) will **decrease** coverage by `3.24%`.
> The diff coverage is `47.61%`.

> :exclamation: Current head 71152d9 differs from pull request most recent head a6c9676. Consider uploading reports for the commit a6c9676 to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/oom-ai/oomstore/pull/845/graphs/tree.svg?width=650&height=150&src=pr&token=C59L7LTRM4&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai)](https://codecov.io/gh/oom-ai/oomstore/pull/845?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai)

```diff
@@            Coverage Diff             @@
##             main     #845      +/-   ##
==========================================
- Coverage   52.88%   49.64%   -3.25%     
==========================================
  Files          75       75              
  Lines        3050     3062      +12     
==========================================
- Hits         1613     1520      -93     
- Misses       1255     1365     +110     
+ Partials      182      177       -5     
```


| [Impacted Files](https://codecov.io/gh/oom-ai/oomstore/pull/845?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai) | Coverage Δ | |
|---|---|---|
| [internal/database/dbutil/schema.go](https://codecov.io/gh/oom-ai/oomstore/pull/845/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai#diff-aW50ZXJuYWwvZGF0YWJhc2UvZGJ1dGlsL3NjaGVtYS5nbw==) | `67.18% <30.00%> (-6.96%)` | :arrow_down: |
| [internal/database/offline/bigquery/join.go](https://codecov.io/gh/oom-ai/oomstore/pull/845/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai#diff-aW50ZXJuYWwvZGF0YWJhc2Uvb2ZmbGluZS9iaWdxdWVyeS9qb2luLmdv) | `80.32% <63.63%> (-4.16%)` | :arrow_down: |
| [internal/database/online/cassandra/purge.go](https://codecov.io/gh/oom-ai/oomstore/pull/845/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai#diff-aW50ZXJuYWwvZGF0YWJhc2Uvb25saW5lL2Nhc3NhbmRyYS9wdXJnZS5nbw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [internal/database/online/cassandra/get.go](https://codecov.io/gh/oom-ai/oomstore/pull/845/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai#diff-aW50ZXJuYWwvZGF0YWJhc2Uvb25saW5lL2Nhc3NhbmRyYS9nZXQuZ28=) | `0.00% <0.00%> (-88.75%)` | :arrow_down: |
| [internal/database/online/cassandra/import.go](https://codecov.io/gh/oom-ai/oomstore/pull/845/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai#diff-aW50ZXJuYWwvZGF0YWJhc2Uvb25saW5lL2Nhc3NhbmRyYS9pbXBvcnQuZ28=) | `0.00% <0.00%> (-61.12%)` | :arrow_down: |
| [pkg/oomstore/join.go](https://codecov.io/gh/oom-ai/oomstore/pull/845/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai#diff-cGtnL29vbXN0b3JlL2pvaW4uZ28=) | `36.18% <0.00%> (+1.28%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/oom-ai/oomstore/pull/845?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/oom-ai/oomstore/pull/845?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai). Last update [39adc8d...a6c9676](https://codecov.io/gh/oom-ai/oomstore/pull/845?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=oom-ai).


Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 37d77999..5ef98abd 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -51,6 +51,11 @@ jobs:
       oomplay-version: '0.4.4'
     steps:
     - uses: actions/checkout@v2
+    - uses: satackey/action-docker-layer-caching@v0.0.11
+      continue-on-error: true
+      with:
+        key: docker-cache-{hash}
+        restore-keys: docker-cache-
     - uses: actions/setup-go@v2
       with:
         go-version: '1.17'
diff --git a/internal/database/metadata/types.go b/internal/database/metadata/types.go
index a07c8fe0..91cabfcc 100644
--- a/internal/database/metadata/types.go
+++ b/internal/database/metadata/types.go
@@ -2,6 +2,13 @@ package metadata
 
 import "github.com/oom-ai/oomstore/pkg/oomstore/types"
 
+type RevisionRange struct {
+	MinRevision   int64
+	MaxRevision   int64
+	SnapshotTable string
+	CdcTable      string
+}
+
 // Create
 type CreateEntityOpt struct {
 	types.CreateEntityOpt
diff --git a/internal/database/offline/types.go b/internal/database/offline/types.go
index f1450f5f..edab9296 100644
--- a/internal/database/offline/types.go
+++ b/internal/database/offline/types.go
@@ -3,6 +3,7 @@ package offline
 import (
 	"bufio"
 
+	"github.com/oom-ai/oomstore/internal/database/metadata"
 	"github.com/oom-ai/oomstore/pkg/oomstore/types"
 )
 
@@ -13,18 +14,11 @@ type ExportOpt struct {
 	Limit         *uint64
 }
 
-type RevisionRange struct {
-	MinRevision   int64
-	MaxRevision   int64
-	SnapshotTable string
-	CdcTable      string
-}
-
 type JoinOpt struct {
 	Entity           types.Entity
 	EntityRows       <-chan types.EntityRow
 	FeatureMap       map[string]types.FeatureList
-	RevisionRangeMap map[string][]*RevisionRange
+	RevisionRangeMap map[string][]*metadata.RevisionRange
 	ValueNames       []string
 }
 
@@ -32,7 +26,7 @@ type JoinOneGroupOpt struct {
 	GroupName           string
 	Category            types.Category
 	Features            types.FeatureList
-	RevisionRanges      []*RevisionRange
+	RevisionRanges      []*metadata.RevisionRange
 	Entity              types.Entity
 	EntityRowsTableName string
 	ValueNames          []string
diff --git a/pkg/oomstore/join.go b/pkg/oomstore/join.go
index b804d677..ea22c76e 100644
--- a/pkg/oomstore/join.go
+++ b/pkg/oomstore/join.go
@@ -42,12 +42,12 @@ func (s *OomStore) ChannelJoin(ctx context.Context, opt types.ChannelJoinOpt) (*
 	}
 
 	featureMap := buildGroupToFeaturesMap(features)
-	revisionRangeMap := make(map[string][]*offline.RevisionRange)
+	revisionRangeMap := make(map[string][]*metadata.RevisionRange)
 	for groupName, featureList := range featureMap {
 		if len(featureList) == 0 {
 			continue
 		}
-		revisionRanges, err := s.buildRevisionRanges(ctx, featureList[0].Group)
+		revisionRanges, err := s.buildRevisionRanges(ctx, featureList[0].GroupID)
 		if err != nil {
 			return nil, err
 		}
@@ -96,8 +96,8 @@ func buildGroupToFeaturesMap(features types.FeatureList) map[string]types.Featur
 	return groups
 }
 
-func (s *OomStore) buildRevisionRanges(ctx context.Context, group *types.Group) ([]*offline.RevisionRange, error) {
-	revisions, err := s.metadata.ListRevision(ctx, &group.ID)
+func (s *OomStore) buildRevisionRanges(ctx context.Context, groupID int) ([]*metadata.RevisionRange, error) {
+	revisions, err := s.metadata.ListRevision(ctx, &groupID)
 	if err != nil {
 		return nil, err
 	}
@@ -109,22 +109,20 @@ func (s *OomStore) buildRevisionRanges(ctx context.Context, group *types.Group)
 		return revisions[i].Revision < revisions[j].Revision
 	})
 
-	var ranges []*offline.RevisionRange
+	var ranges []*metadata.RevisionRange
 	for i := 1; i < len(revisions); i++ {
-		ranges = append(ranges, &offline.RevisionRange{
+		ranges = append(ranges, &metadata.RevisionRange{
 			MinRevision:   revisions[i-1].Revision,
 			MaxRevision:   revisions[i].Revision,
 			SnapshotTable: revisions[i-1].SnapshotTable,
-			CdcTable:      revisions[i-1].CdcTable,
 		})
 	}
-	ranges = append(ranges, &offline.RevisionRange{
+
+	return append(ranges, &metadata.RevisionRange{
 		MinRevision:   revisions[len(revisions)-1].Revision,
 		MaxRevision:   math.MaxInt64,
 		SnapshotTable: revisions[len(revisions)-1].SnapshotTable,
-		CdcTable:      revisions[len(revisions)-1].CdcTable,
-	})
-	return ranges, nil
+	}), nil
 }
 
 func GetEntityRowsFromInputFile(inputFilePath string) (<-chan types.EntityRow, []string, error) {


Test patch:
diff --git a/internal/database/offline/test_impl/join.go b/internal/database/offline/test_impl/join.go
index 4ab565d3..5dea68b8 100644
--- a/internal/database/offline/test_impl/join.go
+++ b/internal/database/offline/test_impl/join.go
@@ -13,6 +13,7 @@ import (
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 
+	"github.com/oom-ai/oomstore/internal/database/metadata"
 	"github.com/oom-ai/oomstore/internal/database/offline"
 	"github.com/oom-ai/oomstore/pkg/oomstore/types"
 )
@@ -189,8 +190,8 @@ func prepareFeatures(oneGroup bool) (types.FeatureList, map[string]types.Feature
 	}
 }
 
-func prepareRevisionRanges(oneGroup bool, stream bool) map[string][]*offline.RevisionRange {
-	basic := []*offline.RevisionRange{
+func prepareRevisionRanges(oneGroup bool, stream bool) map[string][]*metadata.RevisionRange {
+	basic := []*metadata.RevisionRange{
 		{
 			MinRevision:   1,
 			MaxRevision:   15,
@@ -202,7 +203,7 @@ func prepareRevisionRanges(oneGroup bool, stream bool) map[string][]*offline.Rev
 			SnapshotTable: "offline_snapshot_1_2",
 		},
 	}
-	advanced := []*offline.RevisionRange{
+	advanced := []*metadata.RevisionRange{
 		{
 			MinRevision:   5,
 			MaxRevision:   math.MaxInt64,
@@ -214,12 +215,12 @@ func prepareRevisionRanges(oneGroup bool, stream bool) map[string][]*offline.Rev
 		basic[1].CdcTable = "offline_cdc_1_2"
 	}
 	if oneGroup {
-		return map[string][]*offline.RevisionRange{
+		return map[string][]*metadata.RevisionRange{
 			"device_basic": basic,
 		}
 	}
 
-	return map[string][]*offline.RevisionRange{
+	return map[string][]*metadata.RevisionRange{
 		"device_basic":    basic,
 		"device_advanced": advanced,
 	}
diff --git a/internal/database/online/cassandra/store_test.go b/internal/database/online/cassandra/store_test.go
index 1f6dd058..44267e88 100644
--- a/internal/database/online/cassandra/store_test.go
+++ b/internal/database/online/cassandra/store_test.go
@@ -55,27 +55,22 @@ func TestOpen(t *testing.T) {
 }
 
 func TestGetExisted(t *testing.T) {
-	t.Skipf("Skip this test until we separate snapshot schema and cdc schema")
 	test_impl.TestGetExisted(t, prepareStore, destroyStore(KEYSPACE))
 }
 
 func TestGetNotExistedEntityKey(t *testing.T) {
-	t.Skipf("Skip this test until we separate snapshot schema and cdc schema")
 	test_impl.TestGetNotExistedEntityKey(t, prepareStore, destroyStore(KEYSPACE))
 }
 
 func TestMultiGet(t *testing.T) {
-	t.Skipf("Skip this test until we separate snapshot schema and cdc schema")
 	test_impl.TestMultiGet(t, prepareStore, destroyStore(KEYSPACE))
 }
 
 func TestPurgeRemovesSpecifiedRevision(t *testing.T) {
-	t.Skipf("Skip this test until we separate snapshot schema and cdc schema")
 	test_impl.TestPurgeRemovesSpecifiedRevision(t, prepareStore, destroyStore(KEYSPACE))
 }
 
 func TestPurgeNotRemovesOtherRevisions(t *testing.T) {
-	t.Skipf("Skip this test until we separate snapshot schema and cdc schema")
 	test_impl.TestPurgeNotRemovesOtherRevisions(t, prepareStore, destroyStore(KEYSPACE))
 }
 
diff --git a/pkg/oomstore/online_query_test.go b/pkg/oomstore/online_query_test.go
index 21a12332..2871cef3 100644
--- a/pkg/oomstore/online_query_test.go
+++ b/pkg/oomstore/online_query_test.go
@@ -220,7 +220,6 @@ func prepareFeatures(isConsistent bool, isAvailable bool) types.FeatureList {
 			ValueType: types.String,
 			GroupID:   1,
 			Group: &types.Group{
-				ID:               1,
 				EntityID:         1,
 				OnlineRevisionID: &revision1,
 				Category:         types.CategoryBatch,
@@ -232,7 +231,6 @@ func prepareFeatures(isConsistent bool, isAvailable bool) types.FeatureList {
 			ValueType: types.Int64,
 			GroupID:   2,
 			Group: &types.Group{
-				ID:               2,
 				EntityID:         1,
 				OnlineRevisionID: &revision2,
 				Category:         types.CategoryBatch,
@@ -244,7 +242,6 @@ func prepareFeatures(isConsistent bool, isAvailable bool) types.FeatureList {
 			ValueType: types.Int64,
 			GroupID:   3,
 			Group: &types.Group{
-				ID:               3,
 				EntityID:         2,
 				OnlineRevisionID: &revision2,
 				Category:         types.CategoryBatch,