Repository: openyurtio/openyurt
Pull request #2096 (2024-07-15 03:39:44)

fix: upload autonomy status correctly
<!--  Thanks for sending a pull request!  Here are some tips for you:
https://github.com/openyurtio/openyurt/blob/master/CONTRIBUTING.md 
-->

#### What type of PR is this?
> Uncomment only one `/kind <>` line, hit enter to put that in a new line, and remove leading whitespace from that line:
> /kind bug
> /kind documentation
> /kind enhancement
> /kind good-first-issue
> /kind feature
> /kind question
> /kind design
> /sig ai
> /sig iot
> /sig network
> /sig storage

/kind feature
#### What this PR does / why we need it:

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
-->
Fixes #

#### Special notes for your reviewer:
<!--
use this label to assign your reviewer
/assign @your_reviewer
-->

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

-->
```release-note

```

#### other Note
<!--
If your current PR is still working in process, start the PR title name with [WIP], such as: [WIP] add new crd for yurt-app-manager
If the PR title name begins with [WIP], OpenYurt-bot automatically adds a do-not-merge/work-in-progress label for your pr 
-->


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) Report
Attention: Patch coverage is `54.23729%` with `27 lines` in your changes missing coverage. Please review.
> Project coverage is 56.35%. Comparing base [(`3713163`)](https://app.codecov.io/gh/openyurtio/openyurt/commit/3713163bb07e957f5a0ffd59e6489beeb53451dc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) to head [(`4090add`)](https://app.codecov.io/gh/openyurtio/openyurt/commit/4090addb7eead6455045df4e872d9a724219573e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio).
> Report is 14 commits behind head on master.

| [Files](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) | Patch % | Lines |
|---|---|---|
| [pkg/yurthub/cachemanager/storage\_wrapper.go](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?src=pr&el=tree&filepath=pkg%2Fyurthub%2Fcachemanager%2Fstorage_wrapper.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio#diff-cGtnL3l1cnRodWIvY2FjaGVtYW5hZ2VyL3N0b3JhZ2Vfd3JhcHBlci5nbw==) | 57.14% | [18 Missing and 3 partials :warning: ](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) |
| [pkg/yurthub/cachemanager/cache\_manager.go](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?src=pr&el=tree&filepath=pkg%2Fyurthub%2Fcachemanager%2Fcache_manager.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio#diff-cGtnL3l1cnRodWIvY2FjaGVtYW5hZ2VyL2NhY2hlX21hbmFnZXIuZ28=) | 20.00% | [4 Missing :warning: ](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) |
| [pkg/yurthub/proxy/autonomy/autonomy.go](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?src=pr&el=tree&filepath=pkg%2Fyurthub%2Fproxy%2Fautonomy%2Fautonomy.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio#diff-cGtnL3l1cnRodWIvcHJveHkvYXV0b25vbXkvYXV0b25vbXkuZ28=) | 60.00% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #2096      +/-   ##
==========================================
+ Coverage   56.09%   56.35%   +0.26%     
==========================================
  Files         186      188       +2     
  Lines       18092    18358     +266     
==========================================
+ Hits        10149    10346     +197     
- Misses       6910     6956      +46     
- Partials     1033     1056      +23     
```

| [Flag](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) | Coverage Δ | |
|---|---|---|
| [unittests](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio) | `56.35% <54.23%> (+0.26%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/openyurtio/openyurt/pull/2096?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=openyurtio).

rambohe-ch: /LGTM

Patch:
diff --git a/pkg/yurthub/cachemanager/cache_manager.go b/pkg/yurthub/cachemanager/cache_manager.go
index 08fca4ce..ee3c4472 100644
--- a/pkg/yurthub/cachemanager/cache_manager.go
+++ b/pkg/yurthub/cachemanager/cache_manager.go
@@ -68,8 +68,8 @@ type CacheManager interface {
 }
 
 type CacheResult struct {
-	ErrorKeysLength int
-	ErrMsg          string
+	Length int
+	Msg    string
 }
 
 type cacheManager struct {
@@ -80,7 +80,6 @@ type cacheManager struct {
 	cacheAgents           *CacheAgent
 	listSelectorCollector map[storage.Key]string
 	inMemoryCache         map[string]runtime.Object
-	errorKeys             *errorKeys
 }
 
 // NewCacheManager creates a new CacheManager
@@ -98,16 +97,15 @@ func NewCacheManager(
 		restMapperManager:     restMapperMgr,
 		listSelectorCollector: make(map[storage.Key]string),
 		inMemoryCache:         make(map[string]runtime.Object),
-		errorKeys:             NewErrorKeys(),
 	}
-	cm.errorKeys.recover()
 	return cm
 }
 
 func (cm *cacheManager) QueryCacheResult() CacheResult {
+	length, msg := cm.storage.GetCacheResult()
 	return CacheResult{
-		ErrorKeysLength: cm.errorKeys.length(),
-		ErrMsg:          cm.errorKeys.aggregate(),
+		Length: length,
+		Msg:    msg,
 	}
 }
 
@@ -427,7 +425,6 @@ func (cm *cacheManager) saveWatchObject(ctx context.Context, info *apirequest.Re
 			}
 
 			if err != nil {
-				cm.errorKeys.put(key.Key(), err.Error())
 				klog.Errorf("could not process watch object %s, %v", key.Key(), err)
 			}
 		case watch.Bookmark:
@@ -496,13 +493,7 @@ func (cm *cacheManager) saveListObject(ctx context.Context, info *apirequest.Req
 			Group:     info.APIGroup,
 			Version:   info.APIVersion,
 		})
-		err = cm.storeObjectWithKey(key, items[0])
-		if err != nil {
-			cm.errorKeys.put(key.Key(), err.Error())
-			return err
-		}
-		cm.errorKeys.del(key.Key())
-		return nil
+		return cm.storeObjectWithKey(key, items[0])
 	} else {
 		// list all objects or with fieldselector/labelselector
 		objs := make(map[storage.Key]runtime.Object)
@@ -526,20 +517,11 @@ func (cm *cacheManager) saveListObject(ctx context.Context, info *apirequest.Req
 			objs[key] = items[i]
 		}
 		// if no objects in cloud cluster(objs is empty), it will clean the old files in the path of rootkey
-		err = cm.storage.ReplaceComponentList(comp, schema.GroupVersionResource{
+		return cm.storage.ReplaceComponentList(comp, schema.GroupVersionResource{
 			Group:    info.APIGroup,
 			Version:  info.APIVersion,
 			Resource: info.Resource,
 		}, info.Namespace, objs)
-		if err != nil {
-			for key := range objs {
-				cm.errorKeys.put(key.Key(), err.Error())
-			}
-		}
-		for key := range objs {
-			cm.errorKeys.del(key.Key())
-		}
-		return nil
 	}
 }
 
@@ -599,11 +581,9 @@ func (cm *cacheManager) saveOneObject(ctx context.Context, info *apirequest.Requ
 
 	err = cm.storeObjectWithKey(key, obj)
 	if err != nil {
-		cm.errorKeys.put(key.Key(), err.Error())
 		klog.Errorf("could not store object %s, %v", key.Key(), err)
 		return err
 	}
-	cm.errorKeys.del(key.Key())
 	return cm.updateInMemoryCache(ctx, info, obj)
 }
 
diff --git a/pkg/yurthub/cachemanager/storage_wrapper.go b/pkg/yurthub/cachemanager/storage_wrapper.go
index be29e3c0..541e8ef3 100644
--- a/pkg/yurthub/cachemanager/storage_wrapper.go
+++ b/pkg/yurthub/cachemanager/storage_wrapper.go
@@ -19,6 +19,7 @@ package cachemanager
 import (
 	"bytes"
 	"fmt"
+	"strings"
 	"sync"
 
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@@ -44,23 +45,28 @@ type StorageWrapper interface {
 	ListResourceKeysOfComponent(component string, gvr schema.GroupVersionResource) ([]storage.Key, error)
 	ReplaceComponentList(component string, gvr schema.GroupVersionResource, namespace string, contents map[storage.Key]runtime.Object) error
 	DeleteComponentResources(component string) error
-	SaveClusterInfo(key storage.ClusterInfoKey, content []byte) error
-	GetClusterInfo(key storage.ClusterInfoKey) ([]byte, error)
+	SaveClusterInfo(key storage.Key, content []byte) error
+	GetClusterInfo(key storage.Key) ([]byte, error)
 	GetStorage() storage.Store
+	GetCacheResult() (int, string)
 }
 
 type storageWrapper struct {
 	sync.RWMutex
 	store             storage.Store
+	errorKeys         *errorKeys
 	backendSerializer runtime.Serializer
 }
 
 // NewStorageWrapper create a StorageWrapper object
 func NewStorageWrapper(storage storage.Store) StorageWrapper {
-	return &storageWrapper{
+	sw := &storageWrapper{
 		store:             storage,
+		errorKeys:         NewErrorKeys(),
 		backendSerializer: json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme.Scheme, scheme.Scheme, json.SerializerOptions{}),
 	}
+	sw.errorKeys.recover()
+	return sw
 }
 
 func (sw *storageWrapper) Name() string {
@@ -75,6 +81,10 @@ func (sw *storageWrapper) GetStorage() storage.Store {
 	return sw.store
 }
 
+func (sw *storageWrapper) GetCacheResult() (int, string) {
+	return sw.errorKeys.length(), sw.errorKeys.aggregate()
+}
+
 // Create store runtime object into backend storage
 // if obj is nil, the storage used to represent the key
 // will be created. for example: for disk storage,
@@ -83,21 +93,30 @@ func (sw *storageWrapper) Create(key storage.Key, obj runtime.Object) error {
 	var buf bytes.Buffer
 	if obj != nil {
 		if err := sw.backendSerializer.Encode(obj, &buf); err != nil {
+			sw.errorKeys.put(key.Key(), err.Error())
 			klog.Errorf("could not encode object in create for %s, %v", key.Key(), err)
 			return err
 		}
 	}
 
 	if err := sw.store.Create(key, buf.Bytes()); err != nil {
+		sw.errorKeys.put(key.Key(), err.Error())
 		return err
 	}
 
+	sw.errorKeys.del(key.Key())
 	return nil
 }
 
 // Delete remove runtime object that by specified key from backend storage
 func (sw *storageWrapper) Delete(key storage.Key) error {
-	return sw.store.Delete(key)
+	err := sw.store.Delete(key)
+	if err != nil {
+		sw.errorKeys.put(key.Key(), fmt.Sprintf("failed to delete, %v", err.Error()))
+		return err
+	}
+	sw.errorKeys.del(key.Key())
+	return nil
 }
 
 // Get get the runtime object that specified by key from backend storage
@@ -108,11 +127,13 @@ func (sw *storageWrapper) Get(key storage.Key) (runtime.Object, error) {
 	} else if len(b) == 0 {
 		return nil, nil
 	}
+
 	//get the gvk from json data
 	gvk, err := json.DefaultMetaFactory.Interpret(b)
 	if err != nil {
 		return nil, err
 	}
+
 	var UnstructuredObj runtime.Object
 	if scheme.Scheme.Recognizes(*gvk) {
 		UnstructuredObj = nil
@@ -175,21 +196,27 @@ func (sw *storageWrapper) List(key storage.Key) ([]runtime.Object, error) {
 func (sw *storageWrapper) Update(key storage.Key, obj runtime.Object, rv uint64) (runtime.Object, error) {
 	var buf bytes.Buffer
 	if err := sw.backendSerializer.Encode(obj, &buf); err != nil {
+		sw.errorKeys.put(key.Key(), err.Error())
 		klog.Errorf("could not encode object in update for %s, %v", key.Key(), err)
 		return nil, err
 	}
 
 	if buf, err := sw.store.Update(key, buf.Bytes(), rv); err != nil {
-		if err == storage.ErrUpdateConflict {
+		if err == storage.ErrStorageNotFound {
+			return nil, err
+		} else if err == storage.ErrUpdateConflict {
 			obj, _, dErr := sw.backendSerializer.Decode(buf, nil, nil)
 			if dErr != nil {
+				sw.errorKeys.put(key.Key(), err.Error())
 				return nil, fmt.Errorf("could not decode existing obj of key %s, %v", key.Key(), dErr)
 			}
+			sw.errorKeys.put(key.Key(), err.Error())
 			return obj, err
 		}
+		sw.errorKeys.put(key.Key(), err.Error())
 		return nil, err
 	}
-
+	sw.errorKeys.del(key.Key())
 	return obj, nil
 }
 
@@ -198,6 +225,9 @@ func (sw *storageWrapper) ReplaceComponentList(component string, gvr schema.Grou
 	contents := make(map[storage.Key][]byte, len(objs))
 	for key, obj := range objs {
 		if err := sw.backendSerializer.Encode(obj, &buf); err != nil {
+			for k := range objs {
+				sw.errorKeys.put(k.Key(), err.Error())
+			}
 			klog.Errorf("could not encode object in update for %s, %v", key.Key(), err)
 			return err
 		}
@@ -206,18 +236,43 @@ func (sw *storageWrapper) ReplaceComponentList(component string, gvr schema.Grou
 		buf.Reset()
 	}
 
-	return sw.store.ReplaceComponentList(component, gvr, namespace, contents)
+	err := sw.store.ReplaceComponentList(component, gvr, namespace, contents)
+	if err != nil {
+		for key := range objs {
+			sw.errorKeys.put(key.Key(), err.Error())
+		}
+		return err
+	}
+	for key := range objs {
+		sw.errorKeys.del(key.Key())
+	}
+	return nil
 }
 
 // DeleteCollection will delete all objects under rootKey
 func (sw *storageWrapper) DeleteComponentResources(component string) error {
-	return sw.store.DeleteComponentResources(component)
+	err := sw.store.DeleteComponentResources(component)
+	if err != nil {
+		return err
+	}
+	for key := range sw.errorKeys.keys {
+		if strings.HasPrefix(key, component+"/") {
+			sw.errorKeys.del(key)
+		}
+	}
+	return nil
 }
 
-func (sw *storageWrapper) SaveClusterInfo(key storage.ClusterInfoKey, content []byte) error {
-	return sw.store.SaveClusterInfo(key, content)
+func (sw *storageWrapper) SaveClusterInfo(key storage.Key, content []byte) error {
+	err := sw.store.SaveClusterInfo(key, content)
+	if err != nil {
+		sw.errorKeys.put(key.Key(), fmt.Sprintf("failed to store cluster info, %v", err.Error()))
+		return err
+	}
+	sw.errorKeys.del(key.Key())
+	return nil
 }
 
-func (sw *storageWrapper) GetClusterInfo(key storage.ClusterInfoKey) ([]byte, error) {
+func (sw *storageWrapper) GetClusterInfo(key storage.Key) ([]byte, error) {
 	return sw.store.GetClusterInfo(key)
 }
diff --git a/pkg/yurthub/proxy/autonomy/autonomy.go b/pkg/yurthub/proxy/autonomy/autonomy.go
index b0a082e3..f92d1afa 100644
--- a/pkg/yurthub/proxy/autonomy/autonomy.go
+++ b/pkg/yurthub/proxy/autonomy/autonomy.go
@@ -168,17 +168,17 @@ func (ap *AutonomyProxy) tryUpdateNodeConditions(tryNumber int, req *http.Reques
 
 func (ap *AutonomyProxy) updateNodeConditions(originalNode *v1.Node) (*v1.Node, bool) {
 	node := originalNode.DeepCopy()
-	if node.Annotations[projectinfo.GetAutonomyAnnotation()] == "false" || node.Labels[projectinfo.GetEdgeWorkerLabelKey()] == "false" {
+	if node.Annotations[projectinfo.GetAutonomyAnnotation()] != "true" || node.Labels[projectinfo.GetEdgeWorkerLabelKey()] == "false" {
 		setNodeAutonomyCondition(node, v1.ConditionFalse, "autonomy disabled", "The autonomy is disabled or this node is not edge node")
 	} else {
 		res := ap.cacheMgr.QueryCacheResult()
-		if res.ErrorKeysLength == 0 {
+		if res.Length == 0 {
 			setNodeAutonomyCondition(node, v1.ConditionTrue, "autonomy enabled successfully", "The autonomy is enabled and it works fine")
 			atomic.StoreInt32(ap.cacheFailedCount, 0)
 		} else {
 			currentFailures := atomic.AddInt32(ap.cacheFailedCount, 1)
 			if int(currentFailures) > maxCacheFailures {
-				setNodeAutonomyCondition(node, v1.ConditionUnknown, "cache failed", res.ErrMsg)
+				setNodeAutonomyCondition(node, v1.ConditionUnknown, "cache failed", res.Msg)
 			}
 		}
 	}
diff --git a/pkg/yurthub/server/nonresource.go b/pkg/yurthub/server/nonresource.go
index 215e0be6..9c6dc6eb 100644
--- a/pkg/yurthub/server/nonresource.go
+++ b/pkg/yurthub/server/nonresource.go
@@ -59,7 +59,7 @@ func wrapNonResourceHandler(proxyHandler http.Handler, config *config.YurtHubCon
 
 func localCacheHandler(handler NonResourceHandler, restMgr *rest.RestConfigManager, sw cachemanager.StorageWrapper, path string) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		key := storage.ClusterInfoKey{
+		key := &storage.ClusterInfoKey{
 			ClusterInfoType: nonResourceReqPaths[path],
 			UrlPath:         path,
 		}
@@ -91,7 +91,7 @@ func localCacheHandler(handler NonResourceHandler, restMgr *rest.RestConfigManag
 
 func nonResourceHandler(kubeClient *kubernetes.Clientset, sw cachemanager.StorageWrapper, path string) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		key := storage.ClusterInfoKey{
+		key := &storage.ClusterInfoKey{
 			ClusterInfoType: nonResourceReqPaths[path],
 			UrlPath:         path,
 		}
diff --git a/pkg/yurthub/storage/disk/storage.go b/pkg/yurthub/storage/disk/storage.go
index 1a59d66c..c4373ea4 100644
--- a/pkg/yurthub/storage/disk/storage.go
+++ b/pkg/yurthub/storage/disk/storage.go
@@ -421,50 +421,36 @@ func (ds *diskStorage) DeleteComponentResources(component string) error {
 	return nil
 }
 
-func (ds *diskStorage) SaveClusterInfo(key storage.ClusterInfoKey, content []byte) error {
-	var path string
-	switch key.ClusterInfoType {
-	case storage.APIsInfo, storage.Version:
-		path = filepath.Join(ds.baseDir, string(key.ClusterInfoType))
-	case storage.APIResourcesInfo:
-		translatedURLPath := strings.ReplaceAll(key.UrlPath, "/", "_")
-		path = filepath.Join(ds.baseDir, translatedURLPath)
-	default:
+func (ds *diskStorage) SaveClusterInfo(key storage.Key, content []byte) error {
+	if key.Key() == "" {
 		return storage.ErrUnknownClusterInfoType
 	}
-
+	path := filepath.Join(ds.baseDir, key.Key())
 	if err := ds.fsOperator.CreateFile(path, content); err != nil {
 		if err == fs.ErrExists {
 			// file exists, overwrite it with content
 			if werr := ds.fsOperator.Write(path, content); werr != nil {
-				return fmt.Errorf("could not update clusterInfo %s at path %s, %v", key.ClusterInfoType, path, werr)
+				return fmt.Errorf("could not update clusterInfo at path %s, %v", path, werr)
 			}
 			return nil
 		}
-		return fmt.Errorf("could not create %s clusterInfo file at path %s, %v", key.ClusterInfoType, path, err)
+		return fmt.Errorf("could not create clusterInfo file at path %s, %v", path, err)
 	}
 	return nil
 }
 
-func (ds *diskStorage) GetClusterInfo(key storage.ClusterInfoKey) ([]byte, error) {
-	var path string
-	switch key.ClusterInfoType {
-	case storage.APIsInfo, storage.Version:
-		path = filepath.Join(ds.baseDir, string(key.ClusterInfoType))
-	case storage.APIResourcesInfo:
-		translatedURLPath := strings.ReplaceAll(key.UrlPath, "/", "_")
-		path = filepath.Join(ds.baseDir, translatedURLPath)
-	default:
+func (ds *diskStorage) GetClusterInfo(key storage.Key) ([]byte, error) {
+	if key.Key() == "" {
 		return nil, storage.ErrUnknownClusterInfoType
 	}
-
+	path := filepath.Join(ds.baseDir, key.Key())
 	var buf []byte
 	var err error
 	if buf, err = ds.fsOperator.Read(path); err != nil {
 		if err == fs.ErrNotExists {
 			return nil, storage.ErrStorageNotFound
 		}
-		return nil, fmt.Errorf("could not read %s clusterInfo file at %s, %v", key.ClusterInfoType, path, err)
+		return nil, fmt.Errorf("could not read clusterInfo file at %s, %v", path, err)
 	}
 	return buf, nil
 }
diff --git a/pkg/yurthub/storage/etcd/storage.go b/pkg/yurthub/storage/etcd/storage.go
index 2164f18a..ef3bae02 100644
--- a/pkg/yurthub/storage/etcd/storage.go
+++ b/pkg/yurthub/storage/etcd/storage.go
@@ -514,9 +514,9 @@ func fresherThan(rv string, key string) clientv3.Cmp {
 
 type doNothingAboutClusterInfo struct{}
 
-func (d doNothingAboutClusterInfo) SaveClusterInfo(_ storage.ClusterInfoKey, _ []byte) error {
+func (d doNothingAboutClusterInfo) SaveClusterInfo(_ storage.Key, _ []byte) error {
 	return nil
 }
-func (d doNothingAboutClusterInfo) GetClusterInfo(_ storage.ClusterInfoKey) ([]byte, error) {
+func (d doNothingAboutClusterInfo) GetClusterInfo(_ storage.Key) ([]byte, error) {
 	return nil, nil
 }
diff --git a/pkg/yurthub/storage/key.go b/pkg/yurthub/storage/key.go
index 729740f3..8eb044e7 100644
--- a/pkg/yurthub/storage/key.go
+++ b/pkg/yurthub/storage/key.go
@@ -16,6 +16,8 @@ limitations under the License.
 
 package storage
 
+import "strings"
+
 type Key interface {
 	Key() string
 }
@@ -28,3 +30,21 @@ type KeyBuildInfo struct {
 	Group     string
 	Version   string
 }
+
+type ClusterInfoKey struct {
+	ClusterInfoType
+	UrlPath string
+}
+
+type ClusterInfoType string
+
+func (key *ClusterInfoKey) Key() string {
+	switch key.ClusterInfoType {
+	case APIsInfo, Version:
+		return string(key.ClusterInfoType)
+	case APIResourcesInfo:
+		return strings.ReplaceAll(key.UrlPath, "/", "_")
+	default:
+		return ""
+	}
+}
diff --git a/pkg/yurthub/storage/store.go b/pkg/yurthub/storage/store.go
index 6aad2457..74af96f3 100644
--- a/pkg/yurthub/storage/store.go
+++ b/pkg/yurthub/storage/store.go
@@ -16,14 +16,9 @@ limitations under the License.
 
 package storage
 
-import "k8s.io/apimachinery/pkg/runtime/schema"
-
-type ClusterInfoKey struct {
-	ClusterInfoType
-	UrlPath string
-}
-
-type ClusterInfoType string
+import (
+	"k8s.io/apimachinery/pkg/runtime/schema"
+)
 
 const (
 	Version          ClusterInfoType = "version"
@@ -45,10 +40,10 @@ type Store interface {
 type clusterInfoHandler interface {
 	// SaveClusterInfo will save content of cluster info into storage.
 	// If the content has already existed in the storage, it will be overwritten with content.
-	SaveClusterInfo(key ClusterInfoKey, content []byte) error
+	SaveClusterInfo(key Key, content []byte) error
 	// GetClusterInfo will get the cluster info of clusterInfoType from storage.
 	// If the cluster info is not found in the storage, return ErrStorageNotFound.
-	GetClusterInfo(key ClusterInfoKey) ([]byte, error)
+	GetClusterInfo(key Key) ([]byte, error)
 }
 
 // objectRelatedHandler contains functions for manipulating resource objects in the format of key-value


Test patch:
diff --git a/pkg/yurthub/server/nonresource_test.go b/pkg/yurthub/server/nonresource_test.go
index 2b6e5046..a65d880c 100644
--- a/pkg/yurthub/server/nonresource_test.go
+++ b/pkg/yurthub/server/nonresource_test.go
@@ -85,7 +85,7 @@ func TestLocalCacheHandler(t *testing.T) {
 
 	for k, tt := range testcases {
 		t.Run(k, func(t *testing.T) {
-			key := storage.ClusterInfoKey{
+			key := &storage.ClusterInfoKey{
 				ClusterInfoType: nonResourceReqPaths[tt.path],
 				UrlPath:         tt.path,
 			}
diff --git a/pkg/yurthub/storage/disk/storage_test.go b/pkg/yurthub/storage/disk/storage_test.go
index 4017fb27..9fb8839f 100644
--- a/pkg/yurthub/storage/disk/storage_test.go
+++ b/pkg/yurthub/storage/disk/storage_test.go
@@ -1136,7 +1136,7 @@ var _ = Describe("Test DiskStorage Exposed Functions", func() {
 
 	Context("Test SaveClusterInfo", func() {
 		It("should create new version content if it does not exists", func() {
-			err = store.SaveClusterInfo(storage.ClusterInfoKey{
+			err = store.SaveClusterInfo(&storage.ClusterInfoKey{
 				ClusterInfoType: storage.Version,
 				UrlPath:         "/version",
 			}, []byte(versionJSONBytes))
@@ -1150,7 +1150,7 @@ var _ = Describe("Test DiskStorage Exposed Functions", func() {
 			path := filepath.Join(baseDir, string(storage.Version))
 			err = writeFileAt(path, []byte(versionJSONBytes))
 			Expect(err).To(BeNil())
-			err = store.SaveClusterInfo(storage.ClusterInfoKey{
+			err = store.SaveClusterInfo(&storage.ClusterInfoKey{
 				ClusterInfoType: storage.Version,
 				UrlPath:         "/version",
 			}, newVersionBytes)
@@ -1160,7 +1160,7 @@ var _ = Describe("Test DiskStorage Exposed Functions", func() {
 			Expect(buf).To(Equal([]byte(newVersionBytes)))
 		})
 		It("should return ErrUnknownClusterInfoType if it is unknown ClusterInfoType", func() {
-			err = store.SaveClusterInfo(storage.ClusterInfoKey{
+			err = store.SaveClusterInfo(&storage.ClusterInfoKey{
 				ClusterInfoType: storage.Unknown,
 			}, nil)
 			Expect(err).To(Equal(storage.ErrUnknownClusterInfoType))
@@ -1173,7 +1173,7 @@ var _ = Describe("Test DiskStorage Exposed Functions", func() {
 			path := filepath.Join(baseDir, string(storage.Version))
 			err = writeFileAt(path, []byte(versionJSONBytes))
 			Expect(err).To(BeNil())
-			buf, err := store.GetClusterInfo(storage.ClusterInfoKey{
+			buf, err := store.GetClusterInfo(&storage.ClusterInfoKey{
 				ClusterInfoType: storage.Version,
 				UrlPath:         "/version",
 			})
@@ -1181,14 +1181,14 @@ var _ = Describe("Test DiskStorage Exposed Functions", func() {
 			Expect(buf).To(Equal([]byte(versionJSONBytes)))
 		})
 		It("should return ErrStorageNotFound if version info has not been cached", func() {
-			_, err = store.GetClusterInfo(storage.ClusterInfoKey{
+			_, err = store.GetClusterInfo(&storage.ClusterInfoKey{
 				ClusterInfoType: storage.Version,
 				UrlPath:         "/version",
 			})
 			Expect(err).To(Equal(storage.ErrStorageNotFound))
 		})
 		It("should return ErrUnknownClusterInfoType if it is unknown ClusterInfoType", func() {
-			_, err = store.GetClusterInfo(storage.ClusterInfoKey{
+			_, err = store.GetClusterInfo(&storage.ClusterInfoKey{
 				ClusterInfoType: storage.Unknown,
 			})
 			Expect(err).To(Equal(storage.ErrUnknownClusterInfoType))